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ABSTRACT 


This research provides the first theoretical model — the Intranet Efficiency and 
Effectiveness Model (lEEM) — for the Eamily of Measures approach to measure Web 
activity as well as a holistic framework and multi-disciplinary quality paradigm approach 
not previously derived in viewing and measuring intranet contributions in the context of a 
corporation’s overall critical business requirements. This is accomplished by applying a 
balanced baseline set of metrics and conversion ratios linked to business processes as 
they relate to knowledge workers, IT managers and business decision makers seeking to 
increase value. It also outlines who should conduct these measurements and how in the 
form of a business intelligence team and provides a means in which to calculate return on 
intranet metrics investment (ROIMI) with a common unit of analysis for both aggregate 
and sub-corporate levels through forms of the Knowledge Value Added (KVA) and 
Activity Based Costing (ABC) methodologies. 
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EXECUTIVE SUMMARY 


Under current and projected growth rates of information stored in corporate 
intranets and the increasing need to determine how valuable new portals are in collecting 
and applying information contained to meet specific business needs, employing a method to 
holistically, uniformly and regularly measure improvement and take related actions to 
effectively optimize these portals is of mounting importance. As IT professionals and 
business decision makers seek ways to forge their information into knowledge capital that 
can be leveraged quickly for competitive advantage, they require a model and supporting 
metrics to do so. Across any give corporation today, most intranet portal measurements are 
based almost exclusively on usage statistics - with little or no thought given to design and 
user experience factors - and are applied in a freelance and non-standardized manner, 
providing no meaningful insight into how well intranets help corporations achieve their 
strategic objectives. What has been missing is a comprehensive model and methodology to 
base measurements from logically related groups of metrics which, when measured 
periodically, provide actionable steps to optimize efficiency and effectiveness of intranet 
portals to better bolster key business requirements in pursuit of value. This research 
provides the first theoretical model for the Family of Measures approach to measure Web 
activity as well as a holistic framework and multi-disciplinary quality paradigm approach 
not previously derived in viewing and measuring intranet contributions in the context of a 
corporation’s overall critical business requirements. This is accomplished by applying a 
balanced baseline set of metrics and conversion ratios linked to business processes as they 
relate to knowledge workers, IT managers and business decision makers seeking to increase 
value. It also outlines who should conduct these measurements and how in the form of a 
business intelligence team and provides a means in which to calculate return on intranet 
metrics investment (ROIMI) with a common unit of analysis for both aggregate and sub¬ 
corporate levels through forms of the Knowledge Value Added (KVA) and Activity Based 
Costing (ABC) methodologies. Corporations that use the holistic six segment perspectives 
outlined in the Intranet Efficiency and Effectiveness Model (lEEM) to define, apply and 
refine a balanced set of metrics and conversion rates will measure and take action on what is 
important, instead of what is available. 



THIS PAGE INTENTIONALLY LEET BLANK 



I. INTRODUCTION 


Much relevant work in other disciplines such as finance, economics 
and decision theory are seldom encountered by the typical software engineer. An 
interdisciplinary approach that integrates such work with traditional software 
engineering theory and practice promises real advantages. The purpose of 
software design, as for any design activity, is to create added value. Yet, current 
theories, concepts, tools and methods are not clearly based on modem models of 
value and value creation... 

Manifesto, Economics Driven Software Engineering Research (EDSER) 

A. BACKGROUND 

The corporate portal is one of the earliest broad applications of information 
technology (IT) to immediately impact knowledge management practice. Corporate 
portals have a purpose that is similar to, but fundamentally distinct from, Internet portals 
that have become the popular interface for searching and traversing the World Wide 
Web. Corporate portals provide an architecture and set of technologies with which to 
build a single point of access across a wide variety of applications, repositories, 
processes, and functions that have proliferated in the corporate information environment. 
The portal market can be divided into four broad categories: business to employee, 
business to consumer, business to supplier, and personalized Web front-ends, like Yahoo. 
This paper focuses on portals designed for business to employee, known as intranets. 

Most enterprises today find themselves poorly positioned to take competitive 
advantage of the recent proliferation of corporate information sources resulting from the 
advent of the intranet. A fundamental shortcoming in doing this is due to a lack of 
comprehensive and credible means in which to measure how effective the portals meet 
the demands of their employees (also known as knowledge workers) and other intended 
audiences in pursuit of carrying out business objectives. 

Intranets exist to fulfill different purposes for different constituencies than does 
the Internet. The key difference lies in the underlying mission of the portal itself: on the 
Internet, the portal sites’ business model is based on attracting a portion of the advertising 
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budgets of corporations that might otherwise advertise in other media (print, TV, radio, 
etc.). Thus, the general purpose of the public portals is to attract large numbers of repeat 
visitors, to build online audiences with the inclination to buy what the portal advertisers 
have to sell. These portals have essentially settled into a one-directional relationship with 
their viewers. 

Inside the organization, the portal takes on an entirely different character. It takes 
its purpose from the overall mission of the organization; to add sufficient value for its 
customers to create a sustainable business model. It takes its features and functionality 
from the mandate to operate at world-class efficiency and effectiveness in order to remain 
competitive. Achieving this competitiveness requires a bi-directional model that can 
support the knowledge workers’ increasingly sensitive needs for pertinent, helpful, timely 
content and interactive information management tools. 

The primary functionality of a portal is in its ability to select, filter, expose, and 
deliver with a significant degree of precision the information that knowledge workers 
within an organization require to pursue core business processes and competencies in 
support of strategic business requirements. Unlike publication portals or commercial 
portals which serve essentially as one way bulleting boards, the corporate portal has its 
information context predetermined by the information requirements and applications 
already in place in the organization. In this respect, corporate portals share some of the 
built-in context which gives distinctive shape to departmental and group portals 
supporting knowledge workers within or partners outside who share similar job functions 
and objectives. The opportunity for corporate portal developers is to identify and tap into 
the underlying organization in disparate corporate information, which, if correctly 
exploited, can yield significant benefits for knowledge workers who are tasked with 
navigating the myriad internal and external sources of connections among people, 
processes, and the information sources. As organizations realize their customer data and 
the intellect of their knowledge workers are their greatest assets, intranets will grow and 
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their portals will be better intereonneeted through inereased emphasis on taxonomy and 
tagging management in support of more relative and faster seareh and navigational finds. 
The goal is to provide virtually all the information knowledge workers need at their 
fingertips to be highly produetive and to sustain competitiveness. 

Portals will become a competitive necessity in many sectors without which a 
corporation would be at a serious disadvantage. Content which once was lost in a sea of 
intranet websites, obsolete navigation hierarchies, or legacy data stores will one day be 
integrated into a single and more personalized experience — dramatically increasing the 
quality of results made and thereby improving the odds that more can be accomplished 
with them. This, by inference, leads to increases in productivity and greater value. 
Corporate intranets built today will eventually be logically brought together, optimized 
and deployed correctly making them, in effect, the backbone of the knowledge portals of 
tomorrow. 

B, MOTIVATION 

As organizations continue to consume massive amounts of data, the number of 
portals deployed gradually increases to personalize and meet the needs of specific groups. 
In fact, the primary impetus behind portal adoption is deep user dissatisfaction with a 
one-size-fits-all Internet as well as unwieldy intranets. Usually the larger an intranet 
grows the inherently more inefficient and ineffective it becomes, further inhibiting 
information from becoming knowledge. Nevertheless, there is great potential to 
transform large intranets into productivity applications in their own right and a growing 
number of corporations are using portals to begin turning this into a reality. As a result, 
the critical work of software engineering in applying data abstraction as well as software 
architecture design and support required to build efficient and effective intranets is 
playing an incontrovertible and significant role. 
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The primary objective of creating Intranet portals is to create value through 
quality improvements in productivity and customer satisfaction by the exchange of 
desirable business related knowledge. Recent interest in the software engineering 
community in portal processes to leverage knowledge reflects this aspect of the quality 
paradigm. One fundamentally unique way of seeing and solving problems in the quality 
paradigm is to focus on processes rather than products, services, mistakes, errors, or 
traffic volume. Work in the quality paradigm deals with recognizing, defining, 
measuring, analyzing and improving processes. A central value of the quality paradigm 
is that processes must not merely be improved but they must be continually improved. 
The same holds true in measuring and improving value created by intranet portals. Only 
those organizations that continue to refine and measure the subsequent impacts of their 
portals will realize greater value gains. 

The value portals apparently already bring to enterprises is strongly evidenced by 
the recent surge in portal investments'. Corporations of all kinds expect big benefits such 
as better access to information — but large majorities of them say they do not measure 
intranet benefits because they are too abstract and don't convey the impact that their 
portals have on overall corporate performance. Nevertheless, knowing how intangible 
assets affect performance can mean the difference between growth and erosion of value. 
Intranet analytics is not an oxymoron as perceived by many, but it can be counter¬ 
intuitive and more research should be devoted to this area to lead the way in how to seize 
considerable new opportunities. 

Presently, most IT organizations today do not have clear ways to calculate and 
communicate the direct benefits produced. Similarly, the software engineer community 
lacks deep theoretical understanding of the means by which core concepts of their field 
are linked to value creation, e.g., modularity in design (architecture), iterative 

* According to one research firm, 60 percent of Global 2000 companies will have at least a first-generation 
employee portal this year and most of these portals will be designed primarily for employees' use 
(Anderson, pp. 43-57). 
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development methods, testing, ete. Thus, the eurrent eonceptual state of the art in 
software design and engineering is not elearly optimal for value. To the extent that it is 
not, society is not getting its money’s worth from its software engineers. The emerging 
area of strategic analysis, software and domain design, undertaken in this research, takes 
the value-based view as a premise and seeks to develop and enhance fundamental 
software architecture and design theories, tools and methods by basing them on emerging 
models of value and value creation which are supported by both quantitative and 
qualitative metrics groupings. 


Less - 


More 
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Strategic Management 


Economic Valuation /' 
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Knowledge Workers 


Finance £ 
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Engineering 


Figure 1, Software Engineering Approach to Metrics 


Information technology contributes more than data; it impacts a wide range of 
business processes that produce the required results to sustain value. It is in the analysis 
of these processes that answers to productivity can be derived. However, in most 
corporations there is no consistent, visible and uniform way to measure, manage and 
maximize the effectiveness of IT-enabled business processes of intranets. The motivation 
for this research is to provide a model, {Intranet Efficiency and Effectiveness Model, 
lEEM), and means of measuring the effectiveness of a processes integrated approach to 
quality through the establishment and recommended application of a baseline set of 

metrics and their conversions ratios that collectively and logically relate to key strategic 
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business requirements that drive value. Hypothetically, incremental results are 
achievable. This paper further outlines who should do the analytics and how in order for 
corporations to move forward with a continuous optimization process. 


C. CONCEPTUAL SCOPE OF RESEARCH 

Taking an interdisciplinary approach to this matter reveals the underlying 
segments that constitute an intranet and their dependencies. This can then be delineated, 
resulting in logical constituents that can be measured with groups of related metrics using 
multiple tools and techniques. The supposition is derived metrics that indicate 
effectiveness of a corporation’s intranet portals are achievable and can be presented to 
provide meaningful feedback as well as actionable steps to further improve portal 
operations by taking into account and uniformly applying across all portals, a more 
holistic and coherent combination of metrics from all intranet segments. The end result 
and goal is to drive more value back into an intranet than would be garnered otherwise 
through more precision tuning of portals in support of business requirements based on 
audience needs and behavior as well as usage traffic figures and conversion rates. 

D, RESEARCH CONTRIBUTION 

Although most intranet portal owners agree in principle that measurement is 
important for success, many fail to properly implement a portal management effort unless 
budget justifications are required. Indeed, research indicates that 76% of Global 2000 
firms do not conduct benchmarking even once a year. 

• only 19% of Global 2000 firms conduct it annually. 

• just 14% continually refresh benchmarks and use the results regularly in 
performance reporting (Roth, 2002). 

There is a woeful lack of forethought in selecting metrics for intranets, soliciting focused 
and “impactful” surveys for feedback or how to do appropriate analysis with it to provide 
actionable steps to improve portal effectiveness. Across any given corporation, most 
intranet portal measurements today are based almost exclusively on usage statistics - 
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with little or no thought given to design and user experience faetors - and are applied in a 
freelanee and non-standardized manner. While these metries provide a basis to 
benehmark against eompetitors or other departments/groups within the organization, they 
provide no meaningful insight into how well businesses are aehieving their strategie 
objeetives. 

As more and more portal mangers seek a means to ensure portals are being 
properly deployed and modified in support of their employees, their sponsoring business 
deeision makers (BDMs) will seareh for a strategy to prove its suceess, and not one based 
strietly on eonventional return on investment (ROI). Portal eosts are akin to sunk eosts 
and the actual costs to improve them usually only require analytical capabilities 
supported by minor and affordable modifications to the system and not costly, major 
hardware and software developments that require long stretches of time. 

The most meaningful way to measure an intranet is by the effectiveness of its 
collective portals in support of business requirements. The delineation between costs and 
revenue versus effectiveness and productivity — along with the lack of any 
straightforward model-based method to indicate the increase in the potentiality of events 
that lead to productivity gains — is the reason why measuring intranets are often 
overlooked and misunderstood. The research surrounding the Intranet Efficiency and 
Effectiveness Model contributes to the methodology and analytic basis needed to select a 
baseline group of metrics and conversion ratios that help to logically and holistically 
prove out and optimize effectiveness of intranet portals as they relate to and bolster 
strategic business requirements which steer the productivity factors. An intention being 
corporations can use the six segment perspectives outlined in the lEEM to define, apply 
and refine a balanced set of metrics to begin measuring what is important, instead of what 
is available. 


7 



Though credit is due other perspectives, meeting the business requirements is the 
bottom line to measuring the effectiveness of any website. Within the total set of metries, 
there needs to be some monetary business-based metrics, such as ROI, customer lifetime 
value, customer recency, conversion rates, and cross-portal referrals — depending 
naturally on the specific business objectives of the corporation’s overall eustomer 
relationship and management strategy. To ealculate these metrics, corporations need to 
integrate data from many sources, possibly spanning the complete enterprise. This can be 
quite ehallenging, but in the end organizations need these metrics as tangible feedback for 
their efforts in creating, maintaining and managing their Web portals, partieularly if they 
are required to present IT-related initiatives in finaneial terms for return on investment 
analysis. 

To this end, this research provides the first theoretieal model for the Family of 
Measures approaeh to measure Web activity (Section II.B: Review of Current IT Analysis 
Approaches) as well as a holistic framework and multi-disciplinary approach not 
previously derived in viewing and measuring intranet contributions in the eontext of a 
corporation’s overall eritical business requirements by applying a balanced baseline set of 
metrics and conversion rations — which merge website traffie data as well as user 
behavior - that are linked to business processes as they relate to knowledge workers, IT 
portal managers and BDMs. It also outlines who should do this and how in the form of a 
business intelligence team and further provides a means in whieh to calculate ROI for the 
metries investment with a common unit of analysis for both aggregate and sub-corporate 
levels through a form of the Knowledge Value Added (KVA) and Activity Based Costing 
(ABC) methodologies. 

E, WAY AHEAD 

The following chapters address in sequenee related work and their gaps, the 
conceptual framework of the Intranet Efficiency and Effectiveness Model introduced, 
how the lEEM is related and applied to strategic business requirements, followed by how 



a return on investment can be calculated and by whom, before concluding with a 
summary and a recommendation for future related work. 
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II. ASSESSMENT OF RELATED WORK 


The software development proeess must ineorporate more 
and more strategie aspects. Project success is more about whether the 
software delivers value that's greater than the cost of the resources put into 
it - but that's very tricky to measure. 

Martin Fowler, Enterprise Software Consultant 

A, CURRENT APPROACHES TO WEB MEASUREMENTS 

The primary reason for most investments in information technology is to improve 
business processes. The problem becomes one of discerning how much value the IT will 
add to the processes. One way to answer to this question would be to determine how 
much return the IT provides at the aggregate (referred to hereafter as corporate) and sub¬ 
corporate levels. There have been numerous approaches to assessing the impact of IT on 
company economic performance at the corporate level of aggregation and sub-corporate 
levels (Brynjolfsson and Yang, 1996). The lEEM is the first theoretical framework that 
can comprehensively unify analysis to address this problem, regardless of level of 
aggregation. 

Although a great deal has been written on how to calculate return of investments 
for the Internet, there is an appreciable lack of data on how to measure the effectiveness 
of an intranet. Current IT measurements either focus predominantly on online volume 
traffic or its ROI based on some monetary cost allocation calculation. The following 
table, taken from Models for Measuring the Return on Information Technology by Dr. 
Thomas Housel (Housel et ah, 2001, p. 4), provides a snapshot of some of the 
predominant, current approaches. This proposal, A Metric Model for Intranet Portal 
Business Requirements, is appended at the end to provide a basis of comparison and to 
highlight its strengths over the other methodologies presented. A review of these 
approaches follows which identifies a number of problematic issues and limitations of 
each. 
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Approach 

Focus 

Example 

Level of 
Analysis 

Key 

Assumption 

Key 

Advantage 

Limitation 

Process Of 
Elimination 

Treats effect of IT 
on ROI as a 
residual after 
accounting for 
other more easily 
measurable capital 
investment 

Knowledge 
Capital 
(Strassmann 
2000a, b) 

Aggregate 
corporate -level 
only 

ROI on IT 
difficult to 

measure 

directly 

Uses commonly 

accepted 

financial 

analysis 

techniques and 

existing 

accounting data 

Cannot drill 
down to effects 
of specific IT 
initiatives 

Production 

Theory 

Determines the 
effects of IT 
through input 
output analysis 
using regression 
modeling 
techniques 

Brynjolfsson 
& Hitt (1996) 

Aggregate 
Corporate - level 
only 

Economic 
Production 
Function Links 

IT Investment 
Input To 
Productivity 
Output 

Uses Econo¬ 
metric Analysis 
on Large Data 
Sets to Shows 
Contributions 
of IT at the 

Firm Level 

"Black-Box" 
approach with no 
intermediate 
mapping of IT's 
contributions to 
outputs 

Resource- 
Based View 

Linking Firm Core 
capabilities with 
competitiveness 

Jarvenpaa & 
Leidner (1998) 

Aggregate 
Corporate - level 
only 

Uniqueness of 

IT Resource = 

Competitive 

Advantage 

Strategic 
advantage 
approach to IT 
impacts 

Causal mapping 
between IT 
investment and 
Firm Competi¬ 
tive Advantage 
difficult to 
establish 

Option 

Pricing 

Model 

Determines the 
best point at 
which to exercise 
an option to invest 
in IT 

Benaroch & 

Kauffman 

(1999) 

Corporate/Sub¬ 

corporate 

Timing 

Exercise Option 
= Value 

Predicting The 
Future Value of 
An IT 

Investment 

No Surrogate For 
Revenue At Sub 
Corporate Level 

Family of 
Measures 

Measure multiple 
indicators to 
derive the unique 
contributions of 
information 
technology at the 
subcorporate level 

Balanced 
Score-Card 
(Kaplan and 
Norton, 1996) 

Sub-corporate 

Need Multiple 
Indicators to 
Measure 
Performance 

Captures 
Complexity of 
Corporate 
Performance 

No Common 

Unit of Analysis, 

No Theoretical 
Framework 

Cost-Based 

Use cost to 
detemiine the 
value of 
information 
technology 

Activity-based 

Costing 

Johnson & 

Kaplan 

(1987) 

Sub-corporate 

Derivations of 
Cost =: Value 

Captures 

Accurate Cost 
of IT 

No Surrogate For 
Revenue At Sub 
Corporate Level, 
No Ratio 

Analysis 

Intranet 

Efficiency 

and 

Effective¬ 
ness Model 

Use hard and soft 
combinations of 
metrics from each 
segment of 
intranet 

Business 
Requirement 
to Audience 
Metrics 
(Jacoby 2002) 

Corporate/Sub¬ 

corporate 

Multiple groups 
of logical 
metrics 
associated to 
key business 
requirements of 
productivity 

Provides Theore 
tical ramework 
and Surrogate 
for Revenue At 
Corporate and 
Sub Corporate 
Levels 

Not all 
subjectivity in 
analysis is 
completely 
eliminated 


Table 1, Common Approaches to Measure the Return on IT 
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B, REVIEW OF CURRENT IT ANALYSIS APPROACHES 

Research on the problem of estimating the value added by IT can be categorized 
at two levels of analysis: corporate and sub-corporate. At the corporate level existing 
approaches can be categorized as: Process of Elimination, Production Theory, and 
Resource-based View. Two of the prominent sub-corporate approaches can be 
categorized as: Family of Measures and Cost-Based. The Options Pricing Model and 
lEEM span both levels in that they can be used to assess corporate level IT investments 
or individual IT initiatives. 

1. Corporate Analysis 

The corporate level approaches are largely designed to help investors understand 
the contribution of corporate assets such as knowledge and technology (e.g., see 
Brynjolfsson and Hitt 1996, Brynjolfsson and Yang 1996, Im, Dow, and Grover 2001, 
Strassman 1997) to a firm’s or industry sector’s performance. 

a. Process of Elimination 

In the "process of elimination" approaches, the various costs for capital 
(e.g., equipment, real estate, raw materials) are removed leaving the cost of technology. 
Once the costs for capital are accounted for and income proportionately reduced, the 
residual is asserted to be revenue attributable to knowledge capital and/or information 
technology. Following this approach, all costs attributable to all cost categories, except 
IT, would reduce the income proportionately leaving the income attributable to the IT. 

The limitation of these approaches to measuring the value added by IT to 
core processes is that they only pertain to the aggregate view. It is unlikely that such 
aggregated views would allow precise inferences about performance improvements 
derived from IT initiatives at the process level (Birchard and Nyberg 2001). 
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b. Production Theory 

Others have used eeonomie-based produetion theory to determine the 
various eontributions of inputs to the firm’s output. Hitt and Brynjolfsson (1995) 
measured the value of IT in terms of produetivity, profit, and eonsumer welfare. 
Extending this researeh, Brynjolfsson and Yang (1996) report eomprehensively on IT’s 
effeet on overall eompany performanee. The resulting “produetion funetion” 
(Brynjolfsson and Hitt 1996, p. 545) ean be modeled using eeonomie theory to determine 
the unique eontributions of IT with eomputer eapital, non-eomputer eapital, information 
systems staff labor and other labor expenses as the inputs (whieh represent all spending 
by the organization as well as all eapitalized investment) and output in terms of dollars or 
physieal units. This neoelassieal eeonomie theory of produetion treats firms as “blaek 
boxes” (Brynjolfsson and Hitt 1996, p. 544) and attempts to deduee the relationship 
between inputs and outputs without referenee to aetivities within the eompany. 

The strength of sueh approaehes derives from their relianee on eommonly 
aeeepted finaneial-eoonomie theories and the use of existing aeeounting data. This 
makes them transparent for review and eomparison. However, various eritieisms have 
been leveled at these approaehes ineluding that the researeh using these approaehes does 
not “adequately eontrol for other faetors [i.e., other than information teehnology] that 
drive firm profits” (Bharadwaj 2000, p. 170). Along the same lines, Im et al. (2001, p. 
104) stated, "Beeause many faetors influenee firm performanee, it is difficult to establish 
causality between IT investments and firm-level output performance." This lack of 
intermediate mapping of IT impacts on processes makes this class of approaches 
problematic for providing the kinds of feedback necessary to help management determine 
whether their IT initiatives are really paying off. 

c. Resource-Based View 

Researchers using the resource-based view attempt to overcome the 
shortcomings of the prior methodologies for estimating the value added by IT by 


14 



reformulating the problem. They attempt to link a firm’s performanee to IT resourees 
that are firm-speeific sueh as knowledge, eapabilities and unique core processes 
(Bharadwaj 2000, Jarvenpaa and Leidner 1998) . The assumption is that these unique 
resources and capabilities are difficult, and very expensive, to copy and therefore provide 
competitive advantages leading to superior economic returns. 

A limitation of this view for tracking the specific value-added 
contributions of IT is that it does not posit a common, granular unit of analysis that would 
allow an unambiguous linkage or mapping between a firm’s use of IT and the resulting 
cost-benefits performance. Using this approach, it would be difficult to unambiguously 
determine the specific contribution of a given IT initiative. 

d. Option Pricing Models 

The application of option pricing models (0PM) to IT investment has 
attracted increasing attention (cf Benaroch and Kauffman, 1999). Basically, the 0PM 
approach in evaluating IT initiatives determines the best point at which to exercise an 
option to invest. Applied in the context of real options investments, there are six 
variables used to make the decision: The current value of the underlying asset. 

• The time to the decision date. 

• The investment cost or exercise price (also called the strike price). 

• The risk-free rate of interest. 

• The volatility of the underlying asset, which is often the only estimated 

output. 

• Cash payouts or non-capital gains returns to holding the underlying assets. 
(Amram and Kulatilaka 1999, 37). 

The 0PM approach has some implicit assumptions that are potentially 
limiting. For example, net present value is used in the calculation of risk and requires an 
assumption about projected cash flow. However, there is no cash flow directly 
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attributable to most corporate processes. This is because the outputs of those processes 
are not salable to end customers without the outputs of all the other processes, limiting 
the applicability of this approach at the sub-corporate level. 

2, Sub-Corporate Analysis 

The current aggregate level approaches do not appear to resolve the problem of 
determining the IT impacts on process performance (sub-corporate level) with enough 
precision to benefit managers who must implement changes at the process level. Hence, 
other approaches for addressing problems of measuring the impact of IT have focused at 
the sub-corporate level. 

a. Family of Measures 

The family-of-measures approaches. Balanced Scorecard and Intellectual 
Capital Navigator advocate the need to measure multiple indicators to derive the unique 
contributions of information technology at the sub-corporate level. The “Balanced 
Scorecard” provides typically from four to five key performance indicators selected by 
management to determine the success of a given strategic organizational thrust (Kaplan 
and Norton 1996). In the case of an IT initiative, the organization’s management team 
might select this initiative for assessment via a set of key performance indicators, for 
example including level of customer satisfaction, financial performance, employee 
satisfaction and core process performance. Edvinsson and Malone’s (1997) Intellectual 
Capital Navigator allows a firm to identify up to 140 variables that account for the 
performance of its intangible assets including IT (i.e., a subset of its infrastructure assets). 
Examples of these measures would include: laptops/employee, IT expense/employee, IT 
staff/staff total, IT literacy of employees, and so on (Edvinsson and Malone’s 1997, p. 
109). 
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b. Cost-Based 

Many of the cost-based approaches use underlying replacement cost (e.g., 
transfer pricing, internal markets, outsourcing) to determine the value of information 
technology (Housel and Bell 2001). These approaches assume that the cost of IT is in 
some way proportionate to its value. For example, the cost to replace or outsource IT is 
presumed to be proportionate to the value it adds to process performance. Other 
approaches assume that by introducing a market mechanism where corporate managers 
submit bids for IT services, the resulting market price is representative of the IT’s 
underlying value (Ba, Stallaert and Whinston 2001). 

C. HOW THE lEEM APPROACH WOULD DIFFER AND COMPLIMENT 

The Intranet Efficiency and Effectiveness Model views intranet portal-related 
initiatives and application of its associated metrics baseline and conversion ratios in the 
context of the organization’s overall strategy by linking metrics groups to critical 
business requirements and how they effect and are affected by knowledge workers, portal 
managers and business decision makers. Along these lines, the framework of lEEM 
actually complements and provides the first theoretical model based on all segments of 
the intranet processes for the Eamily of Measures concept. In addition, the metrics 
baseline is similar to that of the Balanced Scorecard but extends it by providing context 
linked to strategic business requirements. The ROI for lEEM (see section V.E: Objective 
of ROIMI) is similar to Cost-Based in that it is founded on cost-benefit analysis with time 
being the common unit of analysis which can be used at both the corporate and sub¬ 
corporate levels, a faculty lacking in the other analysis approaches. 

The main assumption when a corporation assumes the lEEM’s supporting metrics 
to serve as a baseline to perform analytics is that it already has an intranet (i.e., sunk cost 
as a part of doing business) and believes there is room for continuous optimization in it to 
increase its net value. The ROI conjectured is essentially a cost-benefit between the time 
needed to invest in applying and acting upon the results of the metric analytics and the 
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results in time reduetions due to subsequent ehanges introdueed by this analysis proeess, 
i.e., shorter eompletion time of a series of business related tasks (sub-eorporate level) and 
speed to market or completion of a project (corporate level). 

1, Family of Measures Similarities and Differences 

The two of the sub-corporate approaches similar to lEEM are Eamily of Measures 
and Cost-Based. The premise of the Eamily of Measures, like that of lEEM, is that no 
single performance measure can encompass the range of work undertaken by most 
business processes. A range of measures, therefore, must be used which reflects the main 
inputs and outputs of the department. A “family” of measures attempts to measure as low 
as possible to reflect the work of a department in a way no single measure can. Eor 
example, the key performance measures chosen for a management accounts department 
might be: 

• Reports with errors / Total Errors 

• Actual cost / Eorecast Cost 

• Average Report Production Time 

• Reports late / Total Reports 

• Number of reports / Number of staff 

• Average Cost of Reports 

These measures should encompass the range of inputs used: labor, materials, 
capital and equipment, and the measures should encompass the factors of quantity, 
quality, timeliness and cost. A Eamily of Measures such as these, related to the real 
objectives of a department, allows the level of performance to be measured and 
monitored over time to establish whether it is stagnating, declining or improving. It 
allows the effect of productivity initiatives to be established and focuses managers’ 
attention on the need to improve productivity — a true and proven way to improve a 
company's competitive position. 
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Unlike the lEEM and its supporting baseline of metries and eonversion ratios, the 
limitation of these Eamily of Measures approaches is that they do not provide a common 
theoretical framework and consequent common unit of analysis that would tie 
investments in IT unambiguously to a firm’s economic performance (Bharadwaj 2000). 
The lack of a common theoretical framework leads to an inherent problem of subjectivity. 
Though subjectivity is not completely eliminated in the lEEM, analysts and researchers 
of the current Eamily of Measures approaches are left to develop subjective assessments 
of how the variables relate to each other rather than the guidance of a formal framework 
or mathematical model. In most cases, simple normalization to an interval scale is used 
along with an assumption of linearity of the relationship among the variables with 
weightings provided by the subjective judgments of management about the importance of 
the various measures (Housel et ah, 2001, p.9). 

2, Cost-Based Similarities and Differences 

Although there is also a cost-benefit analysis involved in the metrics supporting 
the lEEM, the generally accepted conceptual limitation of the cost-based approaches to 
generating a return on investment-type performance ratio is that they do not have a 
surrogate for revenue (Johnson 1992). The problem of using this method for evaluating 
the value added by IT, is the fact that if cost (or any of its derivatives) is used as a 
surrogate for value, then all the information is contained in one term of the ratio, i.e., the 
denominator. The data source for value should come from the revenue side of the firm's 
performance (i.e., numerator) and the data source for cost (i.e., cost) should come from 
the cost to produce the firm's outputs (Housel et ah, 2001, p.lO). The applicability of one 
popular cost-based technique, known as Activity Based Costing (ABC), to determne ROI 
for the lEEM and its baseline of metrics and conversion rations is examined and 
explained in greater detail in Section V.E.2: Surrogate for Value. 
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D, SUMMARY 

Intranet portals are evolving out of neeessity to better manage data and more fully 
meet the needs of its audienee: the enterprise BDMs, portal owners and managers and 
knowledge workers. Intranets are built to promote effeetiveness and productivity that 
indirectly generate more precision and revenue. Modifications to intranet portals and 
how to measure their efficacy are not exclusively about the immediate and direct return 
of dollars in and of themselves as many ROIs imply. The key is to optimize the portal 
infrastructure that already exists and the processes that take place therein with minimal 
costs. The following lEEM and supporting metrics provide the holistic framework and 
means in which to achieve greater optimization through comprehensive and logically 
model-based quantitative and qualitative cost-benefits analyses which facilitate seeking, 
interpreting and gaining additional value.. 
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III. CONCEPTUAL FRAMEWORK 


Everything should be made as simple as possible, but not simpler. 

Albert Einstein 

A, GENERAL OUTLINE OF APPROACH 

To better appreeiate why portals exist and are occasionally re-designed, one 
should first understand the different domains that support the management of making 
information findable and understandable, their distinct constituents and how these are 
segmented to sustain key business requirements. The lEEM and its underlying baseline 
of metrics were determined by a series of model/diagram procedures that were used to 
break down the composition of an intranet. In order to reveal an abstract domain 
analysis view and the association therein linking metrics groups to critical business 
requirements, an affinity diagram is used to create the conceptual model which separates 
the intranet into distinct and unique segments in order to help map out the 
problem/solution space analysis (see Eigure 2, Intranet Domains and Segments). On top 
of this diagram the various users and their roles are identified within each segment (see 
Eigure 3, Constituent Distribution in Segments and Teams Responsible). Then an 
interrelationship diagram is substituted on top of the affinity diagram to highlight where 
pertinent metrics and their logical relationship between related users and their roles exist 
(see Eigure 4, IDEM Outline of Metric Types and Examples in Each Segment). These 
metrics are further broken down into hard, soft and derived forms and are also outlined 
on the diagrams as well as put into a cause and effect tree table (see Eigure 7, Ranking of 
Key Intranet Metrics by Segment) which shows the impact of these metrics to users as 
well as strategic business values. Easily, the table is put into a prioritization matrix (see 
Table 2, Example of One Metric Breakdown, and for a fuller breakdown Appendix D, 
2 

Domain analysis is a systems analysis for multiple related systems. There is no standard definition of 
domain analysis; several domain analysis methods exist. Common themes among the methods inelude: 

• meehanisms to define the basie eoneepts (boundary, seope, and voeabulary) of the domain 
that ean be used to generate a domain arehiteeture to deseribe the data (e.g., variables, eonstants) that 
support the funetions and state of the system or family of systems. 

• identify relationships and eonstraints among the eoneepts, data, and funetions within the 
domain identify, evaluate, and seleet assets for (re-)use develop adaptable arehiteetures. 
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Intranet Portal Metric Breakdown) to illustrate levels of importance and to establish a 
baseline of metrics in which to commence with measurements. Appendix D is the 
current proposed baseline set of metrics and conversion ratios that result from high-level 
analysis of intranet efficiency and effectiveness factors that gauge and achieve greater 
value and value-creating benefits. 

The lEEM focuses on strategic fit, functional objectives and the opportunity or 
necessity for making process improvements as the keys to success. In addition, the lEEM 
introduces a common theoretical framework that has been missing in which to measure 
ah important facets of intranet processes critical to assessing value. Nevertheless, its 
holistic approach does not eliminate subjectivity altogether as it accounts for critical 
qualitative factors which are often, if not entirely, overlooked in other commonly used 
measuring techniques which concentrate on usage statistics, such as traffic volume. In 
short, as a result of being a more comprehensive model some soft metrics must be taken 
into account which are not strictly quantitative or without human interpretation or 
assumption. Nevertheless, when parties agree upon a relatively well-defined set of 
performance metrics, it is possible to have a relatively unambiguous collaborative 
interpretation of the phenomenon (Baskervihe, 1999, pp. 12). In addition, many of these 
soft metrics can be interpreted quantifiably (see Section IV.B.2: Periodic Soft Metrics). 

Invariably, numerical constructs and identifying assumptions regarding cost 
savings or capital investment and revenue generation are problematic. There is no hard 
and fast rule for defining these assumptions; however, a comprehensive model that 
stresses strategic, functional and technical fit, and seeks to identify opportunities for 
process improvements, is more likely to result in top management support because it is 
based on their direction for the organization, not merely the merits of a particular 
technology. Along these lines, lEEM is particularly well suited to concurrence because it 
systematically takes into account user behavior and feedback which are critical facets to 
understanding and optimizing processes which propel intranet utility. 
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1, Domains and Segments Underlying the Intranets 

The segments outlined below represent six perspeetives of intranet effeetiveness 
measurement, all of whieh have an underlying dependency on each other. Efficiency and 
routing provide the basics that must be maximized before content and design can become 
effective. Content and design may then contribute to overall channel effectiveness which, 
in combination with other channels, provides a healthy bottom line: meeting the business 
requirements. Corporations should take the underlying dependency into account, to make 
sure there is an effective growing path as they phase in their intranet analytics. 


Front-End Ensure Intuitive Navigation 

Quaiity, Authoritativeness, Personaiized 



Back-End 

Develop Shared 
Taxonomy, Global Search 
Across Critical Content 
Domain Integration 


Design 


Content 


Business 

Requirement 


Domain 

Infrastructure 


People, Process, Technology 


Figure 2, Intranet Domains and Segments 


^ The reason for the delineation and the differenee between effieieney and effeetiveness shown above is 
addressed at the end of Seetion III.B, Hard, Soft and Derived Metrics for the Intranet, and again in eontext 
to Return on Investment eonsiderations in seetion V.D.2.a, Quantitative and Qualitative. 
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Web analytics are a vital part of managing any Web portal, but the terminology, 
the technology and its use are far from being mature. A major obstacle that inhibits 
enterprise-level valuations of virtually any kind is the lack of consistent data definitions 
across different portals/groups. Consequently, the lEEM provides a comprehensive 
reference of this sort to alleviate uncertainty and facilitate concurrence (see Appendix E, 
Glossary, for a comprehensive and analytical set of terms oriented for an intranet). 
Similarly, this section following explains the model parts illustrated in Eigure 2 above: 

a. Domains 

Segments and constituents of the intranet fall into one of three domains: 
Eront-end, Back-end and People, Process and Technology: 

(1) Front-End. Eront-end is the processes in which a knowledge 
worker has a more direct interface, i.e., user experience and design or presentation of 
content. 

(2) Back-End. Back-end is the processes that take place that a 
knowledge worker does not see, but can be objectively measured, i.e., by directly 
interpreting server log-file data, packets or server requests. 

(2) People Process and Technology. This domain is where 
challenges of people, processes and technology meet. Eor an enterprise intranet its 
purpose is to develop an improved information finding capability by providing an intranet 
environment that consistently supports both browse and search so that knowledge 
workers can confidently and seamlessly find the technical information they need to work 
effectively and efficiently, thereby increasing value. 

b. Front-End Segments 

The Eront-end domain can be further segmented into two underlying areas 
that knowledge workers directly encounter: 

(1) Design - Editorial Programming. The editor’s experience 
should be a straightforward and consistent use of accepted best practices and best tools 
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available (approved by the company) to organize, access and re-use information in a 
desired manner to produce a desired outcome. 

(2) User Experience. The relationship and cognitive connection 
knowledge worker has with the intranet and the information provided. 

c. Back-End Segments 

The Back-end domain can be further segmented into three underlying 
areas that knowledge workers don’t directly come across while using an intranet: 

(1) Usage. Technical information that relates to or specializes in 
techniques or subjects of applied science. Examples of this include numbers, time and 
duration along with userlD and other work related information, i.e., department, location 
and position. 

(2) Content. All documents — help text, search results, forms, or 
application information — that is delivered to a knowledge worker via a website, or 
application. It is both subject and specific information. For example, “People” is a 
subject and a person by name is specific information. Content is organized and found 
depending on its properties and associations. Being a subjective or soft metric 
measurement, the effectiveness of content is not something that can be evaluated using 
straightforward clickstream analysis (number of clicks to discovering information 
sought). It is not directly quantifiable and requires deeper insight into how the content of 
the website is perceived, for example, through the use of surveys. 

(3) Domain Infrastructure. This is both the physical and software 
interaction representation of the relationships between the key elements within a design. 
It shows how these elements interact and must “transfer” information, so that the 
architect can build the environment. Examples are information maps (document 
elements), server topography, scope of services, permissions (ownership). 


25 



d. Business Requirements Segment 

The focus of this segment is on the outcome of processes which enable 
value. lEEM regards process as the most critical enabler as improvements to it harness 
the merits of technology and provide back the greatest ROI and benefits to people. 

(1) Business Requirements. The over-arching goal of grouping 
metrics within and across segments is to track how well business requirements are being 
supported which promote productivity. Although it is in a separate domain, all segments 
and their constituents from the Front and Back-end domains are designed to support 
critical business requirements (these requirements are outlined in section III.E, Critical 
Business Requirements). 

2, Constituents 

Constituents represent the data necessary in the Front and Back-end domains to 
find information in support of the third domain of People, Process, and Technology, 
specifically those requirements outlined in section III.E, Critical Business Requirements. 
The problem portals try to overcome is the same as the main reason they are created: to 
facilitate knowledge workers’ discovery of all the information they need to do their jobs 
better. This then begs the question of: “What makes information more discoverable?” In 
order for information to be found and useful, portal design must sustain the following 
five vital requisites: 

• Amount of information and meta-information (accessibility & manageability) 

• Understanding information seeking and use behavior 

• Navigation system design used to expose information (logically grouped) 

• Confidence in quality of information and meta-information 

• Relevancy to knowledge worker 

Thus any portal desiring to be part of the information system should contain the 
following 13 constituents of discovering information that collectively sustain the 
requisites above. Eor the sake of simplicity and logic, six of these constituents are 
grouped below into the Back-end as are the remaining seven into the Eront-End domains 
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(examples and a fuller explanation of eaeh eonstituent ean be found in Appendix A, 
Constituents to Finding Intranet Information): 

a. Back-End Constituents 

(1) Content Properties. An integral eonneetion to eontent is its 
eontent properties. How eontent is deseribed in order to affeet the assoeiations made in 
information retrieval and presentation is done through the eontent properties tagged to it. 
The eharaeteristies of a eontent item make up its properties (a.k.a. attributes), sueh as 
author, length, name, ete., ean be represented with a sehema and supported by 
voeabularies of metadata (see Appendix A). 

(2) Domain Information Infrastructure. The domain information 
infrastrueture (DII) of a eorporation is the sum and organization of all its data, 
taxonomies, tools and produets. DII for Web analyties needs to eneompass not all the 
information arehiteeture that is available but only the best elements of these groups that 
ean be further developed and integrated to improve eontrol of eontent and eontext to meet 
knowledge workers needs and to exeeed their expeetations as well 

(3) Domain Integration Framework. This framework is the virtual 
representation of the relationships/strueture between the key elements within a design. It 
shows how these elements internet and must “transfer” information, so that the arehiteet 
ean build the environment. Examples are information maps (doeument elements), server 
topography, seope of serviees, permissions (ownership). 

(4) Information Life Cycle. This eyele refers to events often 
repeated again and again in maintaining the relevanee and aeeessibility of eontent in an 
information system. These events inelude updating, versioning, arehiving and, when 
neeessary, deleting. 

(5) Search. Contrary to other eommon notions of seareh, within 
the lEEM it refers to an application employed by knowledge workers as a tool to find 
through direet surfaeing or through surfaeing an obvious navigational path. 
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(6) User Data. Data comprised of facts and figures a knowledge 
worker maintains private access to for knowledge retention and expansion. This is also 
referred to as “personalization”. 

b. Front-End Constituents 

(1) Accessibility. Information is considered accessible when it is 
available, reachable, and understandable. (Assuming the information exists, ideally 
availability could occur from any point of access of and point of entry to the information 
system. Although how reachable information is depends on a number of factors; in this 
context the more information is compatible with a shared Information Architecture, the 
easier it is to retrieve. How this information is then packaged and presented in a fashion 
that can be understood by the knowledge worker is the last stage of accessibility.) 

(2) Communication of Authoritativeness and Importance. 
Communication of authoritativeness and importance is achieved by communicating to the 
knowledge worker the credibility of an information system within that system to ensure 
confidence and trust in it, i.e., by showing that information comes from a respected and 
newsworthy source within the organization or outside, such as indicating that the article 
is from the Human Resource Department or from a well known national syndicate news 
organization. Another example of this is an organization intranet devoted strictly to 
business related websites. 

(3) Communication of Understanding Search. Communication of 
understanding search is achieved by communicating to the knowledge worker meaning 
and significance of the information they are viewing by keeping it consistent to ensure 
acceptance and engagement, which further increases the audience’s propensity to act on 
information in a common way, i.e., a Glossary of Terms and Definitions, Mouse-over, 
and best practices that are consistently applied to all intranet websites. 

(4) Information Grouping and Segmentation. Information 
grouping is the logical collection of relevant and similar information, such as Content 
Nuggets and Categories respectively. Another aspect of this is information segmenting in 
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which only the relevant parts of a doeument, i.e. a paragraph or two are extraeted and 
placed into fragmented groupings instead of the entire document. For example, first 200 
words of a document returned as part of a search result. 

(5) Navigation - Local and Global. Navigation is a method of 
moving through the domain framework by way visual presentation and consistent 
ehoiees. Navigation has 2 basic types: loeal (also known as vertical) and global (also 
known as horizontal): 

• Local navisation presents ehoiees leading to subtopies or sub-areas of a site, 
usually defined by one of its main menu subjeets (often referred to as a “drill down”). 

• Global navisation presents ehoiees leading to other main areas of a site, i.e. 
Home; Seareh; About. Global navigation is eonsistent throughout an information system 
whieh allows knowledge workers to go across portals with a sense of familiarity. 

(6) Personalization. A method of contextualizing information for a 
knowledge worker based on what is known about the knowledge worker. A site can use 
personalization to alter navigation and eontent presented according to the perceived needs 
of the knowledge worker. Examples include content filtering based on role; 
authentication based on name or status as a manager. 

(7) User Assistance. Help made available to the knowledge worker 
while using an information system. User assistance provides guidance on how to use the 
system or additional help in finding information sought. Examples are help in context, 
feedback. Dialog boxes, and training, real time assistanee from a human intermediary via 
a live ehat. 


The signifieance in Eigure 3 below is the majority of constituents 
(seven) required in sustaining the five vital requisites to find useful information fall into 
the Front-end domain. This is crueial in understanding that the metries taken from this 
end of the intranet must be taken into consideration if portals are to be measured more 
completely with better balanee than they are today. Eailure to do this results in less 
“impactful” actions taken to increase value. 
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Front-End Ensure Intuitive Navigation Quality, 
Authoritativeness, Personalized Experience 

Accessibility 
Navigation 
Customization/ 

Personalization 
User Assistance 
User Data 


Back-End 

Develop Shared Taxonomy 
Global Search 
Across Critical Content 
Domain Integration 



Business Requirement Domain Infrastructure 



People, Process, Techndlo 



Domain Info 
Infrastructure, 




Information 

Grouping 

Navigation 

Personalization 

Search 

Taxonomy 

User Data 


Figure 3, Constituents Distribution in Segments and Teams Responsible 


B, HARD, SOFT AND DERIVED METRICS FOR THE INTRANET 

To measure efficaey of a corporate intranet’s portals requires a coherent and 
balanced combination of metrics taken from all segments. When these metrics are 
collectively and uniformly applied in periodic measurements, they can indicate tractable 
improvements over time. In order to do this, these metrics should be grouped into metric 
categories that support business requirements (see Appendix C, Example of Metrics to 
Measure Intranet Performance, and Appendix D, Intranet Portal Metric Breakdown). To 
this end, there are essentially three types of metrics: hard, soft and derived. 

• Hard metrics can be objectively measured, i.e., by directly interpreting server 
log-file data, server requests, number of visitors over a given period of time, etc. 
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• Soft metrics involve many subjective and qualitative aspects that provide a 
frame of reference to interpret the results, i.e., survey results, visual analysis and 
usability. 

• Derived metrics consist of hard and soft metrics from a variety of business 
and knowledge data involved and a subsequent educated assumption to draw conclusions, 
i.e., estimates of speed to market, value, loyalty and reach. 

With respect to the preceding discussion and the introduction of these metric 
types, a theoretical model (see below Figure 4, IDEM Outline of Metric Types and 
Examples in Each Segment) can be created illustrating the placement of domains, 
segments, constituents as well as a sampling of metric groups that lead to deriving the 
successfulness of portals in supporting business requirements. A distinction can now also 
be made that separates efficiency from effectiveness: efficiency is measured with 
predominantly quantifiable or hard metrics, i.e., numbers and durations of time or both; 
effectiveness is this and more as it takes into account qualitative factors. 


31 



Front-End Ensure Intuitive Navigation Quality, 
Authoritativeness, Personalized Experience 


Accessibility 
Navigation 
Customization/ 
Personalization 
User Assistance 
User Data 


Easier to Measure 


Back-End 

Develop Shared Taxonomy 
Global Search 
Across Critical Content 
Domain Integration 


Content 
Properties 
Info Group. 
Information 
Life Cvcie 



People, Process, Technology 

Challenge and Vision : Given knowledge workers can't find all information 
needed, esp. technical info and experts, with portals that lack consistency in 
management and few central management services in an enterprise intranet 
that is afflicted with duplication/ redundancies and no business intelligence, 
develop an improved information finding capability by providing an intranet 
environment that consistently supports both browse and search so that 
knowledge workers can confidently and seamlessly find the technical 
information they need to work effectively and efficiently. 


Figure 4, lEEM Outline of Metric Types and Examples in Each Segment 


C. AUDIENCES 

An intranet services three distinct audiences: Enterprise business decision 
makers, Portal owners and managers, and Users (or knowledge workers). The interests 
of all three of these audiences must be taken into account and distinguished as much as 
possible to better organize, prioritize and conduct metrics to measure effectiveness of an 
intranet. If people are to be given benefits from enhancements to the enabler of process, 
then it is essential to understand who and where they are as well as the roles they play in 
fulfilling business requirements. 


1, Enterprise BDMs 

Enterprise is the entire operation of a commercially organized company whose 
business activities are aimed specifically at growth and profit through the making of 
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products, buying and selling of goods, and/or providing services. The decisions made by 
high level executives based off what they or their staffs discover in the intranet can have 
a far reaching effect on the other divisions and departments throughout the organization. 
Enterprise BDMs are inevitably responsible for ensuring the company meets its business 
requirements, therefore providing insightful measurements to how the intranet is driving 
value back into the company is a litmus test of great importance to them. 

2, Enterprise Portal Owners and Managers 

Portals are a means into an information system, such as a corporate intranet, 
established over a coherent body of information or community of interest (a.k.a. a site) to 
provide more personalized and relevant information to its users. Examples in an 
organization are a Einance website or HR/Personnel website. Portal owners are usually 
the same as the director or general manager of a large group who directs the creation of 
the group’s own portal to better support its own operations as well as service others. The 
more the information in their portal is leveraged by other portals and users, the more 
valuable their portal becomes to the company - a distinction for which owners strive. 
Portal managers maintain the portals and are responsible for helping the business owner 
meet business requirements through the optimization of the portal itself. Measuring how 
the portal is used in order to improve its performance in this vane is of peak interest to 
both groups of managers, with the preponderance of technical work left to the “hands-on” 
portal manager. 


3, User or Knowledge Worker 

A user is one who gathers, analyzes, adds value and communicates information to 
empower decision-making. The nature of ‘k-work’ is ad hoc, demand-driven and 
creative, both in the ability to create new knowledge greater than the sum of its parts and 
in the ability to present the knowledge in a highly communicative ways (Mattews, 2000, 
p. 39). A critical and mostly overlooked aspect in deriving and prioritizing more 
comprehensive sets of metrics needed to gauge effectiveness is a deeper study of the user 
experience along with more emphasis on specifically what users need in order to do their 
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jobs better, i.e., do users seek technieal information — and if so — is it eode or white 
papers, ete. In this way, portal optimization will gradually provide knowledge workers 
what they need as well as eventually giving them what they didn’t know they needed (as 
eontent management and personalization teehniques mature and beeome more 
sophistieated). 

D, RELATIONSHIPS OF DOMAINS, SEGMENTS, AUDIENCES & METRICS 

When the model is viewed to see how the domains break out with respeet to 
audienees and metries, the following eonelusions stand to reason; Where users internet 
with metries directly occurs in the Front-end. The majority of these metrics are “soft” in 
nature. Examples of popular soft metrics in the Design segment of the Front-end include 
visual analysis, usability and ease of navigation. However, there are a number of hard 
metrics which can be derived to better account for the User Experience segment, such as 
number of mouse clicks to find, time to locate information sought and page abandons. 
Where portal owners are most interested in and what portal managers most interact with 
are metrics in the back-end domain. The majority of these metrics are “hard” in nature. 
Examples of popular hard metrics include top downloads, unique users and duration on 
site. However, there are a number of soft-related metrics in this domain, particularly in 
the Content segment, such as surveys and best bets. Enterprise BDMs have a keen 
interest in the remaining segment. Business Requirements. These requirements are vital 
to the productivity of the company and are what all other segments from inception are 
designed to support and, by extension, what the metrics associated to them as a result of 
the lEEM should ultimately gauge, i.e., reach, loyalty and value. Figure 5 below 
highlights which metrics groups and types fall under each audience as well as how and 
where each corresponds to the Intranet Efficiency and Effectiveness Model. 
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METRICS 



Figure 5, Overlay of Domains, Audiences and Metrics to Segments 

* Usage segment metries represent majority metries used to determine portal performance. 


After the variety of Web metrics are tabulated (see Appendix C: Metrics to 
Measure Intranet Performance), there are approximately 30 times as many metrics in the 
Usage segment as there are in the other segments. Moreover, these metrics are all hard 
metrics. The fact that engineers or people with technical backgrounds, who generally 
conduct metrics in organizations in the first place, tend to favor quantifiable metrics 
along with their simplicity and availability often results in an unbalanced approach to 
measurement, i.e., not taking in periodic and systematic soft metrics, particularly those 
that most impact users from the users perspective. Figure 6 below illustrates the relative 
number of metrics commonly used within each segment. 
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lEEM Metric Proposition : 

Given there are few metrics available in 
other segments outside Usage and given that 
part of the premise of lEEM success is that 
finding information is more effective when 
Font-end factors are better accounted, then a 
more coherent selection of metrics from 
Usage AND from other segments needs to 
be created which take into fuller account 
user behavior and are delineated between 
efficiency and effectiveness for each 
audience (enterprise, portal and user) to 
provide more consistent and meaningful 
measurements that indicate improvement 
gain quantifiably, qualitatively 
and anecdotally. 




METRICS 


Enterprise 


Portals 


Users 


Figure 6, Intranet Segment Metrics Number Comparison 

*Ratio of Usage metrics is 30+ to 1 over all other metrics from all other segment areas. 


Many corporations today focus on routing metrics, such as number of hits per 

page, top 10 search strings, most popular downloads and number of referrals from other 

sources (such as banner advertising, search engines and direct links). These metrics are 

popular and have been created in far greater numbers because they address the issues 

many departments and groups are facing today, namely accessibility and visibility of 

their website. These usage-related metrics are also popular because they can be done 

quickly and are more mathematically straightforward, favoring the proclivity of 

technically oriented workers who usually do the measuring, than the more time 

consuming soft metrics concerned with such things as user behavior and experience in 

the Front-end. In addition, routing metrics are relatively simple to understand at the 
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business level and the data is relatively easy to eollect using the Web server log files. In 
fact, most Web analytics packages provide many routing metrics as pre-packaged reports, 
so it is natural to defer to these “out-of-the-box” tools. Unfortunately, since they are 
often used for Internet websites they are mistakenly applied in like fashion to corporate 
intranets and in a lopsided manner. 

The two audiences with different needs that are driving portal software 
development are the portal managers who want a centralized framework for integrating 
and exposing line-of-business applications, and BDMs who want to make their 
knowledge workers more productive at sharing information and working together. In 
short, IT wants plumbing and BDMs want productivity. However, simple statistics alone 
on plumbing are not going to provide BDMs with all feedback needed to track 
improvements towards productivity. Oddly enough the audience most overlooked in 
ascertaining intranet performance is the user, despite the fact that many of the 
constituents necessary to sustain the requisites of finding information are found in the 
Front-end domain where the knowledge worker resides. Consequently, when selecting 
metrics to measure intranet effectiveness, due consideration needs to be given to metrics 
in the Design and User Experience segments. 

E. CRITICAL BUSINESS REQUIREMENTS 

By determining from all segments which complementary metrics can be coupled 
together in groups and which groups collectively best indicate how well a portal supports 
a business requirement, efficacy indications are attained. It is through the refinement of 
these combinations and groupings of metrics (each organization is unique and therefore 
should take steps to refine their metrics after periodic measurements) from all segments 
of the Intranet that leads to improvement in critical business requirements. The creation 
of these metrics requires a multidisciplinary approach and overview. A good baseline for 
these metrics and their conversion ratios is outlined in Appendix D, Intranet Portal 
Metric Breakdown, and the composition and manner a multidisciplinary team should 
administer and steward these metrics is presented in Chapter VI, The Business 
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Intelligence Team. To this end, the following eritical business requirement terms are 
defined with regard to the quality paradigm’s pursuit for greater value and within the 
eontext of what enterprises strive for to be productive: 

1. Agility 

Agility is the ability both physical and mental to react nimbly and deftly to 
change, i.e., alert combination of speed, skill and intelligence of work and business 
transactions to maintain a competitive edge. Examples of this include the ability to adjust 
to the information needs of a specific audience in order to provide them more 
personalized and relevant data in short order. This can be presented by how well portal 
managers analyze user behavior data taken during visits to sites and then optimize the 
sub-sites, pages and links accordingly. 

2. Disintermediation 

Disintermediation is the removal of parties normally involved in a process, i.e., a 
business transaction between producers and consumers. Examples of this are cross portal 
referrals that allow users to link where they need to be with not having to rely on others 
or begin making phone calls. 

3. Loyalty 

Loyalty is feeling and acting upon a sense of duty out of an attachment to 
something beneficial, i.e., devotion to a particular site that provides useful information to 
get a job done correctly and quickly. This can be indicated, for example, by the number 
of return users and growth of new users. 

4. Opportunity 

Opportunity is an advantageous chance: a chance brought about by coincidence or 
by design through a combination of favorable circumstances. Eor example, modifying 
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the type and number of links on what is determined to be the most visited page should 
present an opportunity for more traffie to links that were less exposed and frequented. 

5. Reach 

Reach is to extend the range of influenee as far as a partieular teehnology and 
serviee allows in order to impaet on people or on a group, i.e., a portal to sehedule and 
promote training wishes to toueh as many potential eustomers within a eompany as 
possible. This can be determined in part by monitoring the number of new and unique 
users from different organizations and roles. 

6. Return on Investment 

ROI is a financial ratio measuring the cash return from an investment relative to 
its cost. Revenue is normally recorded when the product or service is delivered or 
ownership of it changes to the customer. Although costs can be associated to nearly 
every business function (i.e., budgets), costs associated with portals is less clear cut as 
well as the revenue it generates. Therefore, Intranet ROI should be gauged through 
indicators of effectiveness that support the business requirements designed to increase 
productivity. Measuring ROI within the context of this framework is derived from 
analysis taken from and qualified by all metric sub-groups within each portal and 
aggregately across the enterprise (see Chapter V, Return on Intranet Metrics Investment). 

7. Value 

Value in this context is an adequate or satisfactory return on or recompense for 
something of worth, importance and usefulness, i.e., how valuable is the functionality and 
performance of a portal. Although portals impact matters of monetary worth, their value 
expressed in terms of money is of lesser consideration compared to the value they return 
to an enterprise in managing knowledge and facilitating productivity. Similar to ROI, 
value can be derived by reduced time constraints and increased growth in current and 
new audiences. 
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8, Creativity 

Creativity is the ability to use the imagination to develop new and original ideas 
or things, i.e., a new produet line or approaeh to solve a problem. An example eould be 
more eross portal referrals in whieh users of one portal are now direeted to more relevant 
sourees of information in other portals not previously known, thereby inereasing their 
ehanees to be ereative with this wider variety of data along with the subsequent and 
synergistic cognitive connections they will make between them. 

9. Better Decisions 

Better Decisions are all about all corporate employees making better informed 
decisions after considering all pertinent information and choices possible. This metric is 
certainly derived and anecdotal and is essentially a consideration process of a 
combination and culmination of derived metrics which gauge the direction of business 
requirement factors (see Figure 10, Improving and Reducing the Decision Points). 

F. SUMMARY 

In effect, the inter-disciplinary derivation of the lEEM represents a form of data 
abstraction at multiple levels, making it an art as well as a science. It is representative of 
why software engineers who are tasked to do this type of work benefit significantly more 
from multi-disciplinary experiences than they would from predominately one 
background. This holistic approach is essential to determining all segments and their role 
in the human information processes within an intranet. Eor example, defining a business 
by the number of "customers" it possesses is meaningless because loyalty continues to 
elude many sites. Thus, businesses must evaluate themselves with metrics that provide 
meaningful insight into how well they execute their critical business requirements, not 
merely how well they are able to drive visitors and elicit sporadic transactions. The 
lEEM framework helps fulfill this need. Although no single tool addresses all 
perspectives equally well in an efficient manner, corporations need to realize that 


40 



measuring the effeetiveness of aetivity on intranet portals requires periodie applieation of 
multiple tools and teehniques. How to apply the lEEM in light of this is addressed next 
in Chapter IV, Applying the lEEM Framework to Key Business Requirements. 
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IV. APPLYING lEEM FRAMEWORK TO KEY BUSINESS 

REQUIREMENTS 

Most of our intranet is about top-line growth, not the cost reduction 
associated with ROI. Intranet applications often provide new ways of working - 
and how do you measure that? 

Martin Armitage, Director of Global Infrastructure, Unilever Corp. 

A, OUTLINE OF APPROACH 

To date, there has not been a sueeessful demonstration of the methods or strategy 
necessary to successfully implement a measurement technique that can indicate the 
effectiveness of an intranet. By soliciting the three audiences to determine which metric 
area is of greatest impact on their performance and satisfaction (these results can be 
ascertained or confirmed through analysis of personasf, metric areas from each segment 
can then be prioritized, logically grouped and then sub-grouped with specific hard and 
soft metrics supporting each group. All of which are related back to critical business 
requirements and divided among the audience most affected. 

1. Matrix for Metrics and Prioritization 

Figure 7 below shows an initial baseline estimation of the top several metric 
areas, which segment they come from and some of the business requirements they 
sustain. The prioritization of these metrics is based on the metric groups most impactful 
on overall value from a business management perspective. The distribution of metrics is 
of additional interest from a theoretical standpoint because at least one metric area 
originates from every segment. This substantiates the assertion that more metrics need to 
be taken from more than the Usage segment if accurate and comprehensive 
measurements of effectiveness are to be indicated. Nevertheless, it is best to limit the 
number of metrics (the baseline focuses on seven) to include only those that directly 
correlate to a business benefit or else the analysis may become overly complicated, 
risking confusion, implementation and, therefore, credibility. 

If a business fails to understand its users, then it will probably create a poor product or service. Personas 
(or User Archetypes) are a way of depicting users of products, such as user behavior while visiting portal 
sites. Using personas allows development teams to focus more on design and apply appropriate effort. 
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Figure 7, Ranking of Key Intranet Metrics by Segment ^ 

2, Methods to Substantiate Measurements and Theoretical Analysis 

There are three advantages to specifying down to single metrics logically grouped 
(see Table 2, Example of One Metric Breakdown, below for an example and Appendix D, 
Intranet Portal Metric Breakdown, for a far more complete breakdown): 

• When the group consists of hard metrics, the data can be pulled automatically 
and routinely over time, i.e., create, query and retrieve server log fdes supporting metrics. 


^ A derived metric (denoted in the table by “ * ”) can become a hard metric within the Domain 
Infrastructure segment as the perceived physical and software interaction between key elements within 
the constituent of DII becomes concretely known and measured with precision, examples within 
Cross-Portal Reference and Shared Topography above are the ability to capture all unique visitors and 
information maps (akin to server topography) respectively. 
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• Qualitative data can be derived from logically grouped quantifiable data, for 
example, loyalty. 

• When metrics are closely associated to user behavior and business 
requirements, analysis of the results lead to precision and powerful modifications which 
optimize the portal more in line with what needs to be done to improve its effectiveness. 

Table 2, Example of One Metric Breakdown, on the next page presents an 
example of the thought process, reasoning behind it and how to interpret the results. 
Upper portion provides the what, where, who, why and how: 

• What is the metric grouping that has been prioritized to most facilitate 
improvement in critical business requirements. 

• Where locates at the level or segment this metric is most appropriate, for 
example it may apply to the entire enterprise or to mangers of a portal or to user. 

• Who refers to audience(s) most impacted and interested in these results 

• Why outlines the business process and is further delineated into three sub¬ 
categories of Business Issue, Business Question and Business Significance: 

o Business Issue refers directly to the critical business requirement and 
the supporting derived metrics. 

o Business Question relates the main concern from the perspective of 
either or combination of the business decision maker, portal manager or technical analyst. 

o Business Significance is the explanation to Business Questions. 

• How is the best practice of applying the combination of metrics, providing a 
meaningful name for it and then calling out specifically each metric in that group 

In addition, the table is color coded to match the model. All metric areas and specific 
metrics are colored to represent what they support and where they belong in the lEEM as 
they are used with that metric priority or What category. Eor example, the color pink is 
used for all metric areas and metrics that relate to business requirements or are derived, 
yellow for Back-end and hard metrics, blue for Eront-end or soft metrics and purple for 
best bets or practices. 
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WHAT: UNIQUE USERS (Priority No. 3) 

WHERE: Enterprise 

WHO: Portal Owners and Managers 



-Business Req. or 
Derived Metric 
-Back End or 
Hard Metric 

-Front End or 
Soft Metric 

-Best Bet or 
Practice 


WHY HOW 

<-BUSINESS PROCESS-><-BEST PRACTICES- 


— > 
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- 
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- 
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- Visitors Over Time 




vis other sites. 
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- Top Visitors by Hits Over Time 

- Top Visitors by Leads Completed 

- Visits by Length of Visits 

* All of the above for Returning Visitors 

- Mew V5. ReturninM Visited 

Ontimization: 




- 

Lqvu/^ 

What do my visitors come 

Analyze the most popular 

Return Visitor Target Pages tcorrelate return 

- 

Vatii^ 

back for? 

content for my return visitors 

visits with content): 

- 



in order to load and associate 

- Top Returning Visitors by Hits 

- 



related information to meet 

- Top Returning Visitors by Hits Over Time 




demand. 

- Top Returning Visitors by Leads Completed 

- Returning Visitor Visits by Length of Visits 

- Returning Visitor Page Views Over Time 





- Top Document and Content Group for 





Returning Visitors Over Time 





Survey 
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Targeted: 
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Identify where users originate 
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from (specifically, which 

and how your website is 
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- 
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Access 
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particular region organization, 
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group and/or role). 
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- Most Returning Visitors 





Users bv Role 
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- Most Returning Visitors 





Usabilitv Study 





- Internal Visitor Session Activity 

- Usability 


Table 2, Example of One Metric Breakdown (taken from Appendix D) 
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In addition to being grounded in a theoretieal framework, this approaeh ean be 
applied praetieally to obtain estimates based in many eommon units whieh ean be traeed 
direetly baek to speeifie pages, links, design, ete. in a portal. Thus, how to go about 
deriving effeetiveness of portals ean be operationalized in relatively praetieal ways. 
Moreover, this approaeh is not reliant on any partieular software, so it ean be applied to 
any network regardless of its network operation system without additional eosts to 
hardware or software, exeept saving spaee on a server to store queries of log files. 
Additional eosts will be ineurred however in the time eharged by an analyst to eonduet 
the metries or, in the ease of a larger eorporation, the hiring of a full time projeet manager 
to do this work (the extent to whieh these eosts impaet ROI is addressed in Chapter V, 
Return on Intranet Metrics Investments and the eomposition and manner of the analyst(s) 
to do this work is addressed in Chapter VI, Business Intelligence Team). 

B, BRINGING IT TOGETHER 

However, setting a seript to eull data periodieally aeeording to a baseline of 
metries is not suffieient by itself Four eritieal aspeets are required to refine this 
approaeh: 

1. Analysis of User Behavior 

Analysis of user behavior patterns within portals on different sub-sites and pages 
is both informed by the results of metries used and foeused surveys, whieh in turn 
informs and refines subsequent metries and surveys administered. For example, after 
observing that there is a high number of visitors abandoning a partieularly important site, 
user behavior must be taken into aeeount via a foeused survey as well as direet 
observation and analysis of other metries used at this site to aseertain the reason why this 
is oeeurring. In other words, until sound, lower-level algorithms are written to explain 
this behavior, analytieal skills will be required by a knowledgeable “person in the loop” 
analyst (see Seetion VI.B.l: Analyst). 
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Segmentation provided by lEEM along with the attributes that are linked to a 
user’s loginID add to the value of all intranet portal site metries beeause it enables an 
organization to traek disparate behavior in different segment areas by different audienees 
or customer groups. Eor example, determining how first-time users perform differently 
from repeat users is an important indication of how successful trial efforts will be at 
engendering long-term loyalty. Conversion rates and average orders will be lower than 
repeat customers, but narrower gaps will be easier to bridge. Useful comparisons that can 
be made which take into account the combination of user behavior with usage statistics 
are: 

• Percentage of traffic and page views from new vs. repeat visitors 

• Average browse from new vs. repeat customers 

• Conversion rate for first-time visitors and download or browse time 

• Conversion rate for repeat visitors and download or browse time 

• Page views for new vs. repeat customers and visitors 

The end result of analysis may dictate page re-design (i.e., content the portal owner wants 
users to see or believes they need is not being found because it is a few layers deep) or 
new content added (i.e., creating new links or placing content in a more visible location). 
Modifications are then followed by the same set of metrics to see if there is an 
appreciable change in patterns, for instance; more files are downloaded or users are 
staying at the site for longer periods of time than before. 

The metrics must remain consistent or the same between at least two periodic 
measurements to accurately monitor the impact of the process changes made as a result of 
changes to sites. The fewer the changes, the more likely the cause and effect impact can 
be narrowly isolated and tracked. On the other hand, the greater the number of changes, 
the less likely all impacts can be accurately ascertained to the extent tractable value can 
be confidently traced to specific modifications. The application of metrics is a balancing 
act that should lend itself to fewer changes needing to be made over time as the portal 
becomes increasingly optimized. Eigure 8 below. Balancing changes with Cause and 
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Effect, illustrates this point, showing the optimum range of changes that would normally 
occur when first applied to a newer portal and then over time as the same portal becomes 
more optimized. 
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Figure 8, Balancing Changes with Cause and Effect 


Taken a step further after modifications are made, if the metrics indicate users are 
staying at the same site for much greater lengths of time and in greater numbers, either 
additional surveys or metrics may be called for to determine what files are being opened 
and downloaded to better understand and react to what users want. This may be further 
substantiated with clickstream data collection in some instances to observe behavior. 
Although monitoring the every move of an employee may not be desirable, it is certainly 
legal and implications thereof are not as serious when it is administered to employees 
using company portal resources. Once compiled, the results from these metrics could 
affect caching of files and load balance of servers if the demand or user behavior is great 
enough to indicate the server is not configured properly for the type of load it is receiving 
due to (metrics related analysis) enhancements. Alternatively, it could simply mean that 
the site where this behavior is observed in large numbers, needs to be updated and/or 
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optimized to accommodate demand. All in all, the cycle is one of continued refinement 
of portals via metric and survey analysis to meet user behavior, expectations and needs. 

2. Periodic Soft Metrics 

Periodic soft metrics will be required to substantiate and confirm what groups of 
harder metrics indicate. From time to time. Front-end metrics need to be applied, i.e., 
usability studies and surveys from the Design and User Experience segments 
respectively. As suggested in the analysis of user behavior above, surveys can be more 
focused when based on patterns mapped by hard metrics. Surveys based off analysis of 
hard metrics are more succinct and provide desired feedback, increasing the likelihood 
more users will take time to complete them and with less frustration. 

a. Surveys 

A metrics and communication program is critical to both the 
communications process and to the development of a feedback loop so that IT can learn 
which initiatives provide the best business value. If the portal is to succeed as a new 
paradigm for professional computing, it must be able to recognize and adjust to ongoing 
changes in knowledge workers’ information needs - and not solely with usage statistics. 
For example, the portal learning loop in Figure 9 below differs from other architectural 
elements in that it is not concerned with a specific aspect of information management, but 
in the ongoing effectiveness of the portal itself. It enables the portal to adjust 
heuristically to changes in the organizational work and information environment. 

Surveys provide an excellent means to both inform and be informed by the 
learning loop. For example, raw visitors’ metrics might indicate that an infrequently 
visited research page should be archived or discarded, when in fact a single recent access 
may have been the critical piece in securing a major new contract/revenue stream for the 
organization. Benefits from surveys are wide spread and may include other obvious but 
overlooked additions such as providing an online employee manual equipped with a 
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search engine, thereby reducing the amount of time people spend looking for the manual 
and information within the manual (see Appendix B, Online E-Survey Example, for a 
general semi-quantifiable online intranet user e-survey). 
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Figure 9, Portal Learning and Discovery Loop 


Surveys can be administered dynamically and automatically online or the 
old fashioned way with pen and paper. The advantages of conducting surveys online are 
speed and convenience for all parties involved. Provided it is informed and succinct, 
knowledge workers will learn to accept them if they perceive it helps to meet their 
information needs. Surveys that warrant user clarification are far more practical when 
based in conjunction with results from a baseline set of metrics and conversion rations 
that are uniformly applied. Examples include interviewing some users and asking them 
to describe the products (getting their descriptive words) and then observing them 
complete their tasks on the site. Sometimes a survey may ask knowledge workers to 
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name the eategories in which they expect to find the products. At other times a survey 
will look at the pages found at the next level down in the hierarchy to look for potential 
trigger words. These words can then be used to create a multi-level category lists and 
associate them to expand the current taxonomy for broader relevance (within portal 
context) as well as for tagging and storing new corporate information and data formats. 

3, Capture and Apply Lessons Learned 

Lessons learned need to be captured for submission as business rules or best 
practices when methods are proven which optimize portal performance. Optimization 
examples of this kind include: 

• Identifying efficient navigational paths 

• Pinpointing frequently visited areas of the site as potential high payoff areas 

• Planting appropriate keywords within key pages to enhance placement within 
search engines 

• Identifying which referring sites are most effective 

Best practices that result in new policies, regulations and business rules need to be 
supported at the business owner level and higher to ensure recommendations are enforced 
and employed enterprise-wide with a smoother finding experience for knowledge 
workers (see section VI.A, Role of Business Intelligence Team). 

4. A Single Reporting Service 

A single reporting service needs to assist portal managers in administering and 
interpreting the results from the metrics baseline. This could be done by the creation of a 
product manager position (the person-in-the-loop) responsible for driving value back into 
the corporate intranet by maintaining and refining the metrics baseline through analysis 
and regular interaction with both users and portal managers. Without this entity, the 
execution of metrics for the intranet will most likely return to haphazard and freelance 
use of usage metrics focusing on peaks of volume traffic, much like it always has been. 
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To continue measuring intranet performanee through sporadie and often ill- 
eoneeived groupings of metrics — taken almost exelusively from hard metrics in the 
Usage segment — will do little to drive value baek into intranets or provide an aeeurate 
indication of the performanee and effectiveness of portals in support of business 
requirements. In short, it would be near failure to eontinue the status quo in how 
intranets are measured today. A single repository serviee, based on a praetieable model, 
would provide an improvement to eurrent measurement taking teehniques and shed light 
on a greater awareness of the importanee and role of all segments underlying an intranet. 
Some of the benefits that should be aehievable at minimal eost due to this eentralization 
and tighter coordination are; 

• Domain owners experienee inereased traffie by target audienees and higher 
levels of satisfaction with their domain. 

• Domain owners and managers are more effieient with their own resourees 
resulting in a whole that is more effieient than the sum of the parts, i.e., load balanee, 
optimization of most frequently requested pages, downloads, ete. and more effeetive 
editorial modifieations. 

• Domain owners aetively partieipate in the development and adoption of 
standards and best-practices, and are rewarded and reeognized for their work. 

• Enterprise experienees inerease effieieneies (eost savings) due to elimination 
of redundaneies and inereased use of shared serviees, e.g. people databases and 
preseriptive arehiteetural guidanee. 

• Inereased employee satisfaetion with seareh and navigation. 

One other signifieant impaet to a eorporation of a single eolleetion and reporting 
souree is the benefit of being able to direetly develop and refine its own model and 
database from immediate aeeess to data. The hypothesis is that predietors (the metries 
used) get better with praetiee and loeal data provides the best predietive base because it is 
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easier to filter out extraneous variables and to focus on key variables. The issues and 
benefits of a single reporting service are addressed in greater detail in Chapter VI, 
Business Intelligence Team. 

C. PERIODIC REVIEW 

Hard and soft metrics taken together with consideration given to their strengths 
and weaknesses allows an enterprise to make informed decision on the investment in, or 
the ongoing value of its data warehouse and portal system. Achieving success through 
the use of any performance metric will depend as much on how well it is applied as it 
does on when it is used. Studies based on samples and averages over time can make for 
easier and more credible comparisons. Hence, continuous benchmarking should be 
instantiated to confirm and correct baseline measurements and conversion rations through 
periodic (i.e., monthly, quarterly, annually) portal status reviews that measure progress 
against previous baseline results. For these reviews, portal owners should use the metrics 
to determine which roles and content are being underserved by the portal and which 
processes could correct this and better leverage the portals capabilities. 

Three different scenarios would likely play out when an accepted baseline of 
metrics is applied to an enterprise: introductory, intermediate and developed 
implementation. The parties involved (see Chapter 6, Business Intelligence Team) 
determine all relevant metrics possible within the areas that make up effectiveness and 
then prioritize them in importance with regard to relevance, balance, accuracy and 
audience. For example, this work is based in part from data collected from Site Audit 
and Technical User Information Surveys applied against a baseline set of metrics and 
conversion ratios, such as those of lEEM (see Appendix D, Intranet Portal Metric 
Breakdown). 

In the introductory scenario pre and post-pilot tests to measure underlying areas of 
effectiveness using the prioritized, combination of metrics based on lessons learned and 
results from surveys are scheduled to be conducted to find the right fit and rhythm in 
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metrics selected, i.e., how and when they can be most conveniently and operationally 
applied. Afterwards, there needs to be a confirmation meeting with Portal Manager and 
at least one champion BDM to establish agreement with the measurement approach and 
analysis to be conducted as well as to coordinate resources in order to conduct 
measurements and collect/compile data.. After the first pilot is administered to a selected 
group of portals (it may not be wise to apply the pilot to the entire enterprise until 
unforeseen kinks are worked out), data is collected as much by automation as possible 
and then compiled and collated for analysis. Based on the analysis, a few tractable 
changes to processes within the portals are made for further observation with the intent to 
improve productivity. 

Scenario One - Introductory 
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Figure 10, Implementation Scenarios for Applying a Baseline Metrics 


The second post-pilot collection of data then takes place over the same time 
period, usually 30 days, with the same set of metrics and conditions as before, with 
exception of the process modification(s) made. After these results are processed, the 
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analysis may take longer to determine the impact of the changes and the extent to which 
they improved the odds to increase value. In some cases, this analysis may need to be 
facilitated by vendor support until the company is able and willing to do this nature of 
work by itself Following the analysis period, the results need to be present by the 
analysis or business intelligence team to all the portal managers and owners of the portals 
monitored as well as their adjunct BDMs if possible to provide executive level reaction 
and feedback. At the end of this meeting, there should be a wider and better 
understanding of the necessity and impact of a baseline set of metrics and corresponding 
conversion ratios grounded in the lEEM. 

Subsequent measurements need to be taken periodically and preparations for 
improved and expanded runs of the metrics program may eventually become a regular 
job for selected employees at different levels. Over time, less frequent measurements are 
needed as the implementation reaches the developed scenario, whereby measurements are 
taken semi-annually (see Eigure 10 above. Implementation Scenarios for Applying a 
Baseline Set of Metrics). Providing managers at multiple levels with a practical, valid 
and reliable way to monitor portal activity and affects on its processes is only part of the 
battle that needs to be waged by the business intelligence team. Elltimately such 
methodologies must pass muster with the accounting and financial communities and 
those agencies that regulate them (Baskerville, 1999, pp. 46-47). An lEEM baseline 
metric approach would allow collaboration among all concerned parties such that 
interventions based on the use of this methodology could be tracked over time as the 
parties attempt to reach consensus on the meaning of the outputs of such analysis. It is 
only then that such metrics will have a lasting impact. This is precisely why a business 
intelligence team needs to be formed and is discussed at greater length in Chapter VI. 


D, SUMMARY 

The intranet is the most measurable medium ever. With respect to legal issues 
and the compliances more easily demanded within the internal operations for any 
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organization, it is more measurable than the Internet. Yet organizations of all sizes and 
types fail to measure its full impact because it is considered either too hard or not a 
priority. To date, there has not been a successful demonstration of the methods or 
strategy necessary to successfully implement a measurement technique that can indicate 
the effectiveness of an intranet. By soliciting the three audiences to determine which 
metric area is of greatest impact on their performance and satisfaction, metric areas from 
each segment can then be prioritized, logically grouped and then sub-grouped with 
specific hard and soft metrics supporting each group. All of which are related back to 
critical business requirements and divided among the audience most affected. 

As the intellectual capital builds in corporate portals, more investments will be 
made to enhance them and make them greater enablers. Executives will demand from 
their IT that they implement a program of metrics where each major initiative has defined 
goals and metrics to indicate whether or not these goals have been obtained. To translate 
these metrics into financial terms using standard conversion factors, such as the cost to 
the company of each employee saved, the value of time saved, increased revenue per 
customer or transaction, or the savings in time and money from fewer defects will require 
far greater appreciation of intranets and the metrics needed to measure them than is 
exercised in public and private sectors today. Moreover, IT will be required to 
communicate these financially oriented results on a regular basis, something many IT 
professionals today are not versed at doing. The next two chapters on ROI and business 
intelligence teams focus on how to do this and by whom, respectively. 
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V. RETURN ON INTRANET METRICS INVESTMENT 


Return on investment should not be the key benchmark in deciding on a 

company's IT spending. 

Douglas Busch, Chief Information officer, Intel 

A. BACKGROUND 

ROI has become a big buzzword in IT, yet few companies are tracking ROI in a 
consistent and standard manner. Enterprises are increasingly scrutinizing their IT 
spending and proponents of e-business projects must go to much greater lengths to justify 
any spending than they were required to do during the dotcom boom. As a standard 
fallback, companies rely on traditional return on investment (ROI) metrics to make e- 
Business funding decisions. Evaluating the potential return on an IT investment can be 
fairly straightforward—at least in theory. In financial terms, ROI means profit divided by 
investment, expressed as a percentage (King, 2002, p.41). But within that definition, 
there is a lot of room for interpretation and pitfalls: 

• Does the initiative fit strategically? 

• Does the initiative support functional objectives? 

• Does the initiative incorporate opportunities for process improvement? 

• Does the underlying technology fit the infrastructure? 

• Are ROI numbers based on reasonable assumptions? 

By focusing solely on what is quantifiable in terms of dollars and cents to answer these 
types of questions, companies risk being precisely wrong instead of being approximately 
right. The problem being that it is difficult to translate many benefits into hard cash 
equivalents. This applies equally to IT investment in a business context. The tendency is 
to apply a strictly quantitative ROI when in fact true ROI is both quantitative and 
qualitative. Nevertheless, corporate managers, who subscribe to the mantra, "If you can't 
measure it, you can't manage it", want quantifiable statistics to more concretely 
demonstrate that their departmental effort is pulling and financing its share of the weight 
in the corporation. 
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Return on investment is popular because it is a simple concept that everyone can 
understand. ROI and its cousins, Net Present Value, Internal Rate of Return, Payback 
Period, and Economic Value Added are concepts that executives have traditionally used 
to measure performance (see Appendix E, Common Approaches to IT ROI, for 
advantages and disadvantages to these and other techniques de your in calculating ROI for 
IT). These metrics are certainly useful but they fall short of providing a complete 
financial picture for business planning. With Intranet ROI seen as a metric that ranks a 
technology investment in relation to other company investments, attempts are made to 
evaluate intranet expenses in terms of cost savings that are attributable to investments in 
business process automation. ROI therefore would seem like a logical way to assess 
intranet related payoffs. In practice however, traditional ROI metrics fail to measure the 
value created by intranets — forcing business managers at multiple levels to make e- 
Business funding decisions based on gut feelings, rather than tracking ROI in a consistent 
and standard manner with the aid of Web metrics within a business value framework. 

Despite having high expectations for portals, only a minority of firms report 
having formal metrics for documenting portal benefits - and virtually all of these are for 
Internet and not intranet portals. Astoundingly, 51% percent of firms don’t have any 
metrics to prove portal benefits and another 20% don’t know if they have any ROI related 
metrics at all for portals (Gillet, 2001, p. 6). In addition, approximately 66% of IT 
managers believe ROI is an appropriate metric only “sometimes” for an IT site (Upton, 
2001), see Eigure 9, ROI Measured on IT. Moreover, these opinions on the usefulness of 
ROI for IT are based largely on Internet e-Commerce sites, not in relation to intranets 
which are perceived more as “sunk costs” of doing business. All the same, the difficulty 
of determining valid ROI for Internet e-Commerce sites is another reason why IT 
professionals and BDMs often avoid applying ROI to their intranet portals. 
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Figure II, ROI Measured on IT 


I, Intranet Measurement Misconceptions 

If enterprises are to better realize more productivity from their portals, they need 
to understand why it is a fallacy to believe measuring portal value is either undesirable or 
undoable. There are several reasons that perpetuate this false perception; 

• Cost cutting is so much easier to understand and measure than effectiveness 
that it almost always tends to gain prominence, despite a firm's original best intentions. 

• A belief that the accuracy of data in many ROI models is so limited there is no 
point in calculating them. For example, if one is making up numbers to begin with, it's 
not really going to help decide whether an IT project makes sense. 

• Many applications are so inexpensive to develop and deploy that companies 
often assume they'll get a return on their investments or they justify these relatively small 
investments by pointing to intangibles, such as improved employee morale from having 
easy access to their human resources and 401(k) records and better workforce 
collaboration, resulting in quicker time to market. 

• Managers of intranet portals and their bosses generally believe they don't need 
anyone to convince them of its business value. Managers and their knowledge workers 
know that the communities build loyalty, give valuable feedback, and contribute to 
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increased sales. Thus, IT projeet teams shouldn’t jump through finaneial hoops trying to 
eost justify essential investments that are “no-brainers.” 

• Depending on the size of the company, it is not worth the eost or time to 
determine the ROI for an intranet upgrade or enhancement (effectively condueting an 
ROI on the ROI). Though ROI ealeulations are different for different industries and 
eompanies, as a general rule it is not always eost effeetive for small eompanies to conduet 
ROI on modest modifieations to their intranet portals. Many IT organizations go to the 
trouble of doing the ROI math only for expensive projeets. 

Enterprises laek a business strategie model, an affiliated metries baseline and a Business 
Intelligenee Team (BIT) to eonduet the neeessary metries analysis to better diseem and 
balanee appropriate quantitative and qualitative metrics. As a backdrop to the above, IT 
initiatives are inereasingly beeoming so important that eompanies are either not 
evaluating ROI or they are realizing the need to develop new ways to measure ROI to 
take into aecount a projeefs strategie value, i.e., enhaneements to their intranet portals 
and the metries needed to prove and disprove improvement. 

In essence, strategie IT investments that improve the overall effieieney of the 
enterprise will make the business more produetive, saving money and improving the 
responsiveness of the whole eompany. Thus, improving the performanee of an 
enterprise’s intranet portals is of great signifieanee beeause the greatest impact of portals 
is improved produetivity. Consequently, eompanies are going to have to start believing 
in — and doing — this hard work of determining productivity benefits because 
produetivity is the most important pieee of ROI analysis (Koeh, 2002, p. 3).^ Beeause it 
is hard to quantify all intranet-related investment benefits and savings to the business, 
these important faetors get short shrift, even though they ean result in big savings that ean 
have a direet impaet on the profitability and overall effeetiveness of a business. Though 
traditional aceounting methods make it awkward to absolutely quantify IT's value, this 

® In a Darwin survey of 75 CEOs from a broad range of companies, increased productivity was cited as by 
far the most important factor (87 percent, far ahead of the next highest factor, customer satisfaction) they 
use to determine if IT is delivering full value for the money spent. 
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doesn't mean the issue of an IT's value should be ignored. Business owners and finance 
executives alike need to be mindful that ROI methodologies tend to focus on short term 
quantifiable justifications, while ignoring the strategic role IT plays driving new 
opportunities for the business. 

2, Qualitative Inputs and Raw Benefits 

The nature of these metrics-related challenges with respect to value should force 
business owners to find credible ways to rely more on qualitative inputs. As the role of 
IT becomes increasingly strategic to the success of a corporation, a new set of metrics 
needs to be applied to investments that attempt to measure IT's ability to enable increased 
revenue or faster growth for the business. For instance, if an investment improves the 
time to market for a critical new product or service, it could be said to have added 
revenue to the firm. In many situations getting to market early results in a big advantage 
both in terms of more overall revenue in the life of the product or service and in a period 
of higher margins before competitors create downward pressure on prices. In effect, 
portals help meet new challenges to compete on the basis of time — not for the sake of 
speed for its own sake, but because profitability in markets is increasingly available only 
to early entrants who can forge brand and business dominance (a parallel could also be 
made for a government or military in reducing their decision time cycles). 

Return on intranet metric investment (ROIMI) has more credibility and is clearer 
when it's stated in raw benefits, which are sometimes non-quantifiable, rather than 
translated into short-term return dollars. The numbers tell the story, but not the whole 
story: Some benefits may not be quantifiable today in terms of dollars, for example ease 
of use, competitive advantage, customer loyalty, etc. These benefits are worth including 
in the value story, despite the risk of companies still ending up with ROI results that are 
not 100% quantifiable. The challenge is to prevent the translation from becoming fuzzy 
and losing some audiences. 
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Calculating ROI requires a eonsiderable amount of data, eonsistent standards, 
baselines within a eompany, and at least some financial expertise. Even with sueh input, 
the end results are approximate and ean be manipulated. Moreover, intranet ROIs from 
different eompanies are unique and returns may differ because they reflect rapid changes 
in teehnology and knowledge worker behavior whieh varies from eompany to eompany 
as do their expertise levels and fields. Even with these restrictions, the proeedure of 
pursuing ROI, in whatever format, forees a eompany to think about the best ways to 
measure sueeess - speeifieally, the hard, soft and derived metries that matter most in 
measuring the extent to whieh business values are supported and aehieved. 

B, AXIOMS AND DRAWBACKS 

Despite best intensions, ROI ean be misleading and its limitations need to be 
reeognized and addressed. Reeognizing these, a deeper analysis of the IT portfolio ean 
find truly signifieant eontributions to the eorporate health and well-being that should be 
ealeulated and eommunieated. In the past, IT organizations used to be viewed as a 
neeessary expense for a business. Today, however, they are often viewed as an important 
strategie asset to a eompany's future sueeess. While many businesses are foeused on eost 
eontrol, IT groups are often foeusing on how to generate growth for their business by 
funding projects to help their eompany reaeh new eustomers and work more effieiently 
with its existing partners. Perhaps not so surprisingly, only about 8.33 pereent of IT 
spending is pereeived in some eireles to provide ineremental benefit to the organization 
(Gliedman, 2000, p. 2). 

The reality is the benefits often far exeeed this pereeption; otherwise eontinued 
large investments into it would not be made. Eike metries for intranets, intranet ROI ean 
also be eounter intuitive. The following table outlines seven shortcomings that 
companies tend not to take into aeeount when ealculating ROI on IT and its impaet: 
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Axiom 

Drawback 

Impact 

1. ROI must capture all costs, 
direct and indirect, associated 
with the project/technology, 
including products and 

services devoted to direct 
support. 

•One size doesn't fit all. 

•Applying ROI to every activity won't work. 

•The human factor of computer-assisted work is 
neglected by the ROI model. 

•One metric can’t characterize the entire IT value. 

ROI is useless when it comes to evaluating risk, 
flexibility, and intangible benefits-factors that have a 
critical impact on business and should be factored into 
spending decisions. In addition, ROI can’t calculate 
valuable, intangible qualities. 

2. ROI must be based on 
quantifiable results. The 
baseline metric is the dollar, 
often measured through 
calculating time saved 
(time is money). 

•ROI is both quantitative and qualitative. 

Although ROI numbers may appear concrete, they 
can be misleading. Unless ROI analysis is applied 
honestly, there is ample room for delusion. The 
greatest danger is the “concrete” and “measurable”' 
driving the significant out of the analysis. Because 
intranet applications are internally focused, it's 
difficult to get an ROI and is, therefore, neglected. 

3. ROI must be based on 
observable results. 

Automation has business 
value only when we can see 
the results such as faster 
production or higher quality. 

•E-business projects often follow the law of 
unintended consequences because they cross 
functional and enterprise boundaries, and may 
produce payoffs in ways that were hard to imagine 
at the outset. 

Taking an accountant's view of IT priorities could 
actually be counterproductive, because a spreadsheet 
doesn't tell the whole story. In fact, some of the IT 
projects that impact business the most can't be 
measured easily, if at all, some experts now say. 
However, the only way to grow the bottom line on a 
sustainable basis is to grow the top line, which is easy 
to ignore if every project is measured on tangible 
ROI. 

4. ROI emphasizes tangible 
payoffs that can be measured 
in financial terms. Often, the 
easiest to measure returns are 
bottom-line improvements 

rising out of cost reductions. 

•ROI is a metric that favors cost-saving projects. 

ROI tends to favor projects that result in cost 
avoidance, at the expense of projects that promise 
revenue growth. This is particularly acute in ROI for 
intranet initiatives because they are likely to miss at 
the outset subsequent, positive impacts. 

5. Investments in k-worker 
and partner-facing initiatives 
result in more effective 
collaboration and translate 
into important productivity 
benefits for all sides. 

•ROI measures only the returns that the company 
sees within its internal operations. 

By ignoring the value created for partners and 
customers, ROI may be missing the real point of 
e-Business (and the very idea behind creating a 
corporate portal in the first place). For example, as a 
parallel approximately two-thirds of the overall 
benefit of a retail website cannot be accounted for by 
online transactions alone. 

6. ROI calculations for cost¬ 
saving projects are more 
accurate because the 

enterprise already has the data 
needed for the equation. 

•When calculating ROI for a revenue-generating 
project like intranet portals, estimates are often used, 
which makes the ROI calculation less accurate. 

The result is that revenue-generating projects are at a 
disadvantage if they are competing against cost- 
savings projects based on ROI. Furthermore, not all 
data/outputs can be known ahead of time with regard 
to creativity and the volatility of content in data- 
warehouses and the impact of knowledge applied to it. 

7. Most e-business initiatives 
take time to get accepted and 
widely adopted. Declaring 
failure or success is based on 
a three or a six-month time 
period (time is money). 

•There can be significant time lags between the 
benefits that will impact revenue and the revenue 
metrics themselves. The lag time may be six months 
or more for larger projects. Therefore, companies 
examining only the metrics most closely tied to 
revenue risk cutting off projects for which positive 
return may be just around the comer. 

•Not all returns are financial returns in the short mn, 
although they eventually may impact financially. 

Most e-business projects result in payoffs on multiple 
dimensions. It's a tactical approach to ROI. The 
shorter the study, the shorter the ROI basis, the more 
isolated it's going to be and the less meaningful it's 
going to be in the overall strategy. It's significant but 
it needs to be put into the larger context. 


Table 3. Axioms and Drawbacks 
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In effect, in the rapidly changing world of IT, ROI is ROI — except when it’s not. 
While CIOs say the payback on most IT projects can be measured in dollars, many 
utilitarian but necessary efforts, like infrastructure upgrades and installing and supporting 
collaborative applications, don’t translate easily and those projects are not given full 
credit because of the inability (and in some cases, the non-necessity) to attribute any 
intellectual gains to new technology. For example, in the real world financial ROI 
calculations may be 0%, but the overall return of all the measures can easily be well over 
100%. Consequently, the ROI model needs to be extended to be more comprehensive 
and dynamic to take into account time to value while factoring traditional return on asset 
analysis. 

Strictly quantitative ROI logic for intranet portals and IT is not sufficient. For 
example, if every chief financial officer discounted productivity gains, companies would 
not have PCs on their knowledge workers’ desks because they don't have positive ROI 
without the productivity gains and are too costly to manage relative to the hard savings 
they provide. Thus, executives should exercise caution when demanding “balance sheet 
efficiency” on ROI calculations for IT investments. 

Though there are many ways to express standard measures of return, when 
traditional ROI conventions don't tell the whole story, business managers and vendor 
alike will often place greater emphasis on soft benefits — like employee satisfaction, 
improved visibility, improved knowledge transfer, and dozens of other assets that cannot 
be measured in hard numbers — plausibly sure, but with no attempt to put a dollar figure 
on the “smiley faces”. Thus, to overcome this weakness when calculating ROI for IT 
projects, decision makers should consider all techniques available to credibly and better 
measure the overall impact of the investment; they must look beyond ROI. 

C. BEYOND ROI 

Even in some of its strictest applications, ROI is far from being a perfect 
calculation. It is understandable then that an enterprise would be tempted to judge the 
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success of its intranet portal initiatives on its traffie volume statisties alone; they are the 
only value related indieators that ean be exaetly quantified that a portal brings to a 
business, i.e., showing how many knowledge workers visit the site over time and 
therefore providing some level of value. Beeause an intranet does have a measurable 
impaet on traditional businesses, eompanies must embraee a set of metries that gives the 
portal initiatives eredit not only for its online knowledge workers, but also its overall 
eontribution to the eorporation at large in improving its eompetitive advantage. 

The key to eseaping the ROI trap is to think strategieally about the outeomes and 
the payoffs from intranet portals by foeusing more broadly on business value and meeting 
the needs of knowledge workers. The anehor for any e-business projeet should be the 
value ereated. Foeusing on value ereated for eustomers as opposed to eost savings for the 
eompany by supplementing speeulative fmaneial outeomes (some of whieh are dubiously 
eontrived) and quantitative metries with qualitative ones that are rationally more strategie 
in nature (and eolleetively provide more important leading indieators to gauge the 
eompetitiveness of the business) is fundamental in looking beyond ROI. Measures sueh 
as customer and partner satisfaetion, eustomer loyalty, response time to eompetitive 
aetions and improved responsiveness are examples of these soft measures. Subjeetivity 
in these “intangible” measures ean aetually be quite objeetive if used eonsistently over 
time. For example, eustomer satisfaetion measured eonsistently on a five-point seale 
survey can be an objeetive basis for measuring the performanee of eustomer-faeing 
initiatives (Sawhney, 2002, p. 37), see Appendix B, Online e-Survey Example. 

A variety of ROI teehniques exist for managers to assess the value of intangible 
benefits. "Business value added" and "intangible value" are both eoneepts used to 
deseribe how IT dollars support key business goals that aren't easily quantified. 
Similarly, "return on opportunity" helps eompanies examine top-line growth potential 
rather than foeusing on eost savings. "Return on relationship" aeknowledges the 
intangible nature of an e-business by measuring whether relationships produee direet or 
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indirect returns to a company, such as speed-to-market. By contrast, strict financial ROI 
approaches, though straightforward, can easily stifle opportunities to create competitive 
advantage and ignore the impact of intangible benefits altogether. 

Best-practice organizations realize that outcomes are more important than outputs. 
Tracking the interplay between pain points in processes and the subsequent impact of 
modifications to intranet portals to affect business in them based on metrics that tie back 
to key business drivers is the most critical yet underused measure to improve 
performance outcomes. Metrics must tie back to the original business drivers so more 
credible and comprehensive justifications can be provided when the intranet project is 
proposed. Consequently, executive-level ROIs should emphasize outcomes rather than 
hit rates, which is the norm today. New intranet applications often provide new ways of 
working, and companies should approach ROI in this area with sensible notions of “Does 
the opportunity justify the investment” or “Is perceived value greater than the cost?” 
Alternatively, intangible costs and raw benefits can often be quantified by measuring the 
consequences of not making the investment in question; money saved versus the 
prospects of what is to be gained. Thus, in the final analysis, ROI methodology for 
intranet initiatives is more a cost-benefits analysis (see SectionV.F.2.b; Activity Based 
Costing for how this can be accomplished in the form of process changes through metrics 
analysis based on the lEEM). 

Standard definitions of ROI today are beginning to stray from their original 
meaning as input metrics are changing to accommodate increasingly dynamic 
environments such as intranets. Continuing to focus on cost and savings is an operational 
management contribution to “business management” but it doesn’t give satisfactory, 
complete answers about the contribution of intranet portals to an enterprise’s value. 
These answers can only be found at higher levels - at the level of the strategic 
management and economic valuation. Benefits may be expressed in many ways, but the 
key is to express them in understandable terms that focus on value by matching the 
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critical business requirement issues to the needs and inputs of knowledge workers and 
business mangers in the value ereation proeess down to groups of speeifie metrics that 
can be linked and measured in support of these issues and needs within a eoherent and 
eomprehensive framework. The lEEM and its baseline metrics and conversion ratios are 
designed to provide this approaeh to these linkages and to provide insight on how to take 
eorrective aetion upon them. The ehoiee executives faee is not whether an approaeh like 
this should be taken, but whieh groups of metries to ehoose and how to proeeed applying 
them (see Seetion IV.C: Periodic Review). 

D, APPLYING WEB ANALYTICS 

The most important benefit of ealeulating ROI is that the proeess helps determine 
whieh metries are most pertinent to a partieular business. Prom the outset, eompanies 
need to identify and stick to a good starting point when taking measurements to ensure 
what is being measured at the beginning is also being measured at the end in a systematic 
and comprehensive manner. This is partieularly true in measuring the performanee and 
impaet of intranet portals sinee so little has been previously researehed and praeticed in 
how to measure them. Unfortunately, most organizations do not have good data for their 
intranet ROI. One of the main reasons is beeause they don’t have an adequate and 
eoherent eolleetion of “before” data from eaeh segment of their intranet operation 
proeess. Very few eompanies take eomprehensive snapshots of where they are and 
equally few take eomprehensive snapshots of where they are going with respect to overall 
strategic and value-based objeetives. 

When building the ROI justifieation for intranet investments, speeifie metries 
must be predetermined that ean be used to properly analyze and report the neeessary 
information. These metries will be traeked over time so that they ean be mapped to 
aetionable solutions related to bolstering business requirements that will prove out the 
eorreetness of the original business justifieation of the projeet, initiative, or enhaneement 
as well as the solution itself In addition, the ROI methodology and set of assumptions 
must be used in a consistent format to better enable quantifieation of changes in growth 
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and usage patterns. Rather than metrics portraying what happened, a dedicated, 
collective effort to gather this information and analyze it helps to determine what to do 
next to improve performance (see Chapter VI, Business Intelligence Team). Thus the 
process of measuring and fine tuning performance intranet portal impacts caused by 
actions based on metrics analysis is an economic value creator. 

1. Pre-Determined Metrics 

Enterprises need to predetermine the metrics they will collect to assess their 
critical business value objectives, i.e., targeting customer loyalty, partner assessment, 
content effectiveness, channel efficiency, etc. ROI projections for portal framework 
deployments, though useful for project approval, do not provide insight into the real and 
actualized value derived from the portal. As a result, planning the business case for 
portal investments will require predetermined operational metrics, both quantitative and 
qualitative, to be tracked over time. Otherwise, these metrics provide no meaningful 
insight into how well businesses are reaching their strategic objectives. 

The introduction of the Intranet Efficiency and Effectiveness Model (lEEM) in 
this paper outlining how to breakdown, analyze and gauge the impact of changes made in 
support of critical business requirement issues all the way down to specific groups of 
metrics makes it possible to measure the effectiveness of an organization’s quality drive 
toward greater value. It accounts for a variety of factors and indicators that avoid the 
problem associated with isolating the specific impact of any one of those factors on an 
overall conversion rate. A single indicator does not gauge the dial measuring overall 
success but a collection of the right groups of indicators and the metrics to measure them 
can achieve this task over time. These indicators and the metrics all work in concert to 
drive an accurate conversion rate. The lEEM is appropriately broad based to tractably 
gauge and subsequently provide enhancements to dozens of Web-related initiatives that 
have an impact on conversion rate because the constituents in the model take into 
account, for example, the following factors: 
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• Navigation 

• Site layout 

• Site authoritativeness 

• Prequalifieation and disposition of visitors 

• Site performance 

• Scale 

• Speed to fulfillment 

In addition, information flood and false alarms are essentially prevented by defining 
multiple metrics to describe business activities. Consequently, an alert is triggered only 
if a combination of metrics shows certain behaviors. Even if a single metric tells the 
whole story, it is better to have two metrics linked to the resultant alert, or another means, 
such as reviewing earlier analysis, to crosscheck that a problem really exists before 
alerting anyone. 

2. Intranet Analytic Omissions and Susceptibilities 

The following analytic pitfalls in conducting Intranet ROI are outlined to 
highlight the differences and subtleties that need to be accounted for when assessing and 
measuring the value added from intranet processes. 

a. Quantitative and Qualitative 

There are some caveats that need to be addressed for both soft and hard 
metrics with respect to intranets. Dotcoms counted the number of “eyeballs” driven to a 
virtual storefront, but time has shown that the quality of website hits and a site's ability to 
retain customers, known as stickiness or “recency”, is a better measure of ROI and 
business value than measuring site traffic. Quantitative metrics in and of themselves can 
be misleading for intranets and therefore should be supplemented with e-surveys to fill in 
qualitative information that Web logs cannot provide. Because some critical information 
gleaned from usage data or analysis of intranet Web logs is inadequate for measuring 
ROI, the quality of the knowledge worker experience can be improved by implementing a 
feedback loop consisting of regular reviews of quantitative and qualitative metrics. For 
example, a page can have high traffic because the content is uniquely interesting, it 
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represents a gateway to other seetions, or knowledge workers are stuek in a frustrating 
eireular navigation. Similarly, some desirable attributes, sueh as ease of navigation, 
relevanee of seareh results, elarity of eontent as well as the layout of the site, ean only be 
assessed and disambiguated by users. Thus, quantitative metries must be eorrelated with 
qualitative assessments to formulate a eomplete pieture of the user experienee, sueh 
foeused surveys whieh ean be aeted upon, i.e., ehanges to the layout, links and visibility 
of data. 


b. Hit Counts 

Quantitative metries like popular “hit eounts” are most eommonly used for 
intranets today beeause they are readily available and easier to ealeulate. Onee gleaned 
from Web logs however, they present a number of ehallenges to deeision makers when 
used for intranets: 

• The number of hits and the level of produetivity ean be inversely proportional. 
Organizations moving from a eomplex statie intranet to an employee portal often find 
that the number of hits goes down beeause less surfing is needed to find relevant 
information. For example, the portal eould do the surfing for the user based on a speeifie 
user profile, thereby bringing the information direetly to the user via another eontent 
provider/department portal. 

• When the number of hits is used to justify additional modifieations based on 
traffie volume, a low number ean often tell a better story than a high number (e.g., “We 
need more money or manpower beeause we’re not getting hits and therefore need to 
provide more valuable information.”). This ean oeeur as a result of faeilitating the 
delivery of what knowledge workers are seeking through personalization, i.e., plaeing 
speeifieally frequented links on the desktop homepage. 

• Intranets and department portals generate a base level of hits even if they are 
never used, due to auto-starting (with morning boot-up or every time a browser window 
is launehed) and multiple eounting (e.g., portals generating multiple hits for eaeh page as 
“portlets” are rendered). Portal owners should ealeulate the base level of hits that will be 
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incurred automatically and subtract this figure from total hits to generate the number of 
live hits, whieh is a more useful measure of user involvement. 

Hit counts out of context are of limited use. Organizations need to tie hit data back to a 
role- and proeess-based eontext, e.g., matching hits with profiles to determine whieh roles 
are not being served by the portal and which functionality is most used for eaeh role, 
determining how often a particular task is accomplished through the portal. 

Without thorough analysis, even the simplest metrie indieators, like hit 
counts, can be misleading. Failure to reeognize this will degrade portal performance as 
wasted time and effort in implementing changes based on incorreet interpretations. It 
may be something small, sueh as a navigational loop, but it may be corporate-wide 
affecting tens of thousands. Thus, even the most straightforward hard metrics should not 
be taken for granted, but meticulously scrutinized in short order. Using the lEEM 
approaeh over time, triggers that alert analysts to these potential pitfalls will be well 
instated because hit counts will be colleeted with a variety of other metries, such as e- 
surveys, which will aid in identifying pain points. Hard and soft metrics analyzed 
together with consideration given to their strengths and weaknesses allows analysts to 
identify incongruent analysis. Thus, aehieving success through the use of any 
performanee metric will depend as much as how well it is applied as it does on when it is 
used (see Chapter VI, Business Intelligence Team and Section IV.C: Periodic Review, 
respectively for how to realize this). 

c. Conversion Rates 

Though counter-intuitive, since so many faetors impaet on the eonversion 
rate, monitoring a eonversion rate does not enable businesses to determine the precise 
impact of any one faetor. Conversion rate measures ostensibly how effective the site is at 
eonverting its visitors to browse, download, ete. Hence, it needs to be taken into 
eonsideration eolleetively in a coherent manner that eovers a variety of pertinent factors 
from each process segment, whieh the lEEM from ineeption is designed to address. 
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Essentially, a portal seeking to expand its reach and loyalty should deploy initiatives that 
convert their base of registered users and first-time visitors into loyal repeat customers: 
Focus on what drives customer loyalty and higher conversion rates will follow. The 
opposite is not true despite much attention being devoted to devising new conversion rate 
techniques to manipulate the numbers rather than what factors can push them upwards. 

Although efforts at cost reductions can be fairly easily applied throughout 
a firm, efforts aimed at increasing effectiveness generally cannot, unless the same model 
and set of metrics are applied uniformly across all key portals in an enterprise. For 
example, what makes one employee satisfied or productive may not have the 
same impact on another employee. However with the use of identical metrics 
complemented with occasional surveys, data anomalies are mitigated with periodic 
samplings over time because studies from samples and averages based on the same 
method are easier to compare. In spite of the challenges they may present, it is important 
to maintain a balanced approach while pursuing soft benefits — such as customer 
satisfaction and understanding, market intelligence and knowledge transfer — because 
they contribute to middle benefits (derived metrics) — such as speed-to-market and 
loyalty conversion ratios — which directly impact the hard numbers that build a 
company's bottom line. 

d. High-End Knowledge Workers 

The best ways to approach this exercise is by letting key knowledge 
workers express what is useful and believable or not. The payback is great since even a 
small increase in the effectiveness of a firm's most critical workers can impact the firm's 
bottom line. One study estimates that improving the performance of general knowledge 
workers adds about ten times more to the bottom line than facility and IT cost reductions 
combined (Cantrell, 2001). For a firm's most important, "high-end" knowledge workers, 
this ratio is bound to be dramatically higher, cases in point being a software firm or 
research division. Instead of employing a common compromise for all, an enterprise 
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should consider solutions more oriented toward effectiveness solutions for some, and 
solutions more oriented toward efficiency solutions for others. Thus, when selecting 
enhancement solutions, analysts and BDMs alike should bear the following in mind; 
Effeetiveness solutions which tend to be more intangible and soft should foeus on high- 
end knowledge workers (get them involved in the feedbaek loop) and effieieney solutions 
should target employees who eontribute less to a firm's revenue, sueh as administrative 
assistants. 


e. Knowledge Workers ROI Fallacy 

With regard to different types of enhaneements for different levels of 
knowledge workers. Capers- Jones estimates produetivity gains of 50-75% are possible 
(primarily for software and researeh firms) by using outstanding programmers and 
analysts (Casper, 1986). The first measurement of this kind was the Saekman's 
Experiment in whieh large individual differenees were found to exist between 
programmers (Saekman 1968). Another study of this kind eondueted 20 years later, 
known as Demareo's Coding Wars, found similar results but not as dramatie (Demareo, 
1999, p. 27). Table 4 below shows their research results between more and less 
proficient programmers in an organization given the same amount of time to program: 


Saekman’s Results 

Demareo's Results 

Debug Hours : 

18-1 

Best people will outperform the worst by 10:1. 

Code Hours : 

15-1 

Best performer will be 2.5 times better than the median. 

Program Size : 

6-1 

The top 1/2 will outperform the bottom 1/2 by 2:1. 

Run Time : 

13-1 



Table 4, Programmer Productivity Results 


A cursory conclusion to this is that organizations should focus their portal 

efforts on aecommodating their high-end knowledge workers with all the means 

neeessary to do their jobs better. Although this is not entirely ineorreet, it overlooks two 

important faetors with respeet to ROE 1) less skilled programmers do not get paid 10 

times less salary; and 2) there are normally fewer high-end programmers than the lower- 
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end knowledge programmers that work on any given projeet (due to a variety of 
eireumstanees sueh as promotions over time of the more experieneed programmer to 
management positions). Thus, eorporations need to exereise caution when allocating 
resources and prioritizing portal enhancements. The payback may be greater for features 
or changes that affect a wider body of knowledge workers and programmers who are 
considered median or lower-end than for a smaller high-end group. This of course will 
be decided on a case-by-case, or portal-by-portal basis, and is why dynamically 
constructed portals designed to meet the needs of like users is a powerful new 
development in the IT world. All the same, analysts and BDMs must keep in mind 
knowledge worker economies of scale when parceling resources to enhance productivity. 

f Knowledge Workers and Reuse 

The combined and logical approach of model based selected hard and soft 
metrics can lead to better identifying and understanding what knowledge workers are 
doing with what they discover in intranet portals as well as quantifiable and favorable 
ROT For example, as a result of hard and soft metric analysis, ROI may appear in some 
unexpected places such as reuse of software code. Software reuse is a very measurable 
and desirable as it allows cost per function delivered to be dramatically reduced. “For 
instance, a 1,200-member IT team at a Cleveland-based financial firm cut its average 
project turnaround time by an astonishing 45% after it discovered software in a 
development team’s portal that would suffice for other internal projects. On average, a 
single software component took 200 hours to design, at a chargeback rate of $74 per 
hour, or $14,800 per component. When one component was reused in eight different 
projects, it saved the company more than $100,000” (Frakes, 2003, p. 31). Auditing Web 
log files alone would not have captured this, portal quality tracking complimented with 
user feedback resulted in spotting and analyzing ROI returns that would have normally 
gone unrecognized and unaccounted. 
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With respect to reuse, a promise of software and intranets is lower costs or 
at least getting more “bang for the buck”. Focusing on code improvement through portal 
optimization is highly desirable because it can result in both higher reliability and faster 
time to market. However, software reuse until recently has been noticeably missing. 
Although there is promise of turning this phenomena around with the advent of C# within 
.NET and free Linux libraries online, many knowledge workers do not even know the 
code they are writing has already been written or something very similar to it can be 
modified in its place - even when it exists on their own intranet. If made an objective, 
properly configured portals will facilitate software reuse and, more important, knowledge 
reuse, by exposing it and providing such things as references to design documents used to 
implement the module should it be included in other software design templates. Using 
the same metrics groups over and over again across the enterprise to gauge performance 
is itself also a form of reuse provided by lEEM domain analysis and metrics baseline (the 
key to reusable software is captured in domain analysis in that it stresses the reusability 
of analysis and design, not code). Collectively, these are example of how a best practice 
can become a business rule whereby virtually all code must be made accessible along 
with clear understandable documentation. If a business rule is not possible, then an 
incentive program can be devised to reward and recognize portals whose code or 
documentation is most widely and highly used by other co-knowledge workers. 

E. OBJECTIVE OF ROIMI 

Used properly, Web analytics can provide significant returns in optimizing portal 
configurations and capabilities if based on a model that accounts for critical business 
requirements, measured consistently and periodically to determine where refinements are 
needed in order to keep in step with the dynamic needs of users. This is exactly why the 
lEEM has been created so these refinements can be conducted in a logical and coherent 
manner as they impact on critical business requirements, namely to increase value. 
Figure 12 below provides a theoretical illustration why a portal optimized using focused 
metrics is more effective and inevitably more productive than one that does not. 
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Time 


0 

" "N _ 

A 


Information gathered at a point in time 
Consideration of information over time 
Decision to take Informed Action 



Slope of Decision Points to take Informed Action 
Portal that undergoes regular optimization 
Portal that undergoes little or no optimization 


Figure 12, Improving and Reducing the Decision Points 


Given the time it takes to find information A in time t‘ in an a portal that is not 
optimized, a portal that is regularly measured to take into aeeount user aetivity and 
behavior is more optimized to meet users’ needs and, therefore will render more desired 
relevant information ^4+5 in the same time period (point 7 versus point 6). On the other 
hand, if time is of the essence, information A can be found in less time (point 5), thereby 
allowing the user more time to consider the information or to come to a decision sooner 
(point 6 versus point 2). Either way the optimized portal will provide as much relevant 
information in a shorter time period or more information in the same time period. The 
result is either a business decision sooner or a more informed one respectfully. Another 
phenomenon is the near exponential affect of this event over time with regard to an 
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increase in relevant information and the deeision process itself. Though time is 
truncated, it may not always be exponential as the discovery of relevant information can 
be as a result of better association and placement of documents and information based on 
user behavior. Moreover, time spent deliberating to reach a decision may also be 
truncated by a significant margin in some cases since more relevant information is found 
providing clarity on what courses of actions are more likely than others. 


At any given point in time, there will be as more information found in a portal 
consistently using metric analytics based on the lEEM, resulting in more informed 
business decisions being reaehed in less time. In theory, this is how intranet portals 
facilitate speed to market. Table 5 below, lEEM Analysis Impact on Time to Reach 
Decision, is an extension of Figure 12 above, showing the differenee between intranet 
portals without the guidanee of lEEM metrics versus those managed with them: 


Best 

Portal Type 

Time Info 
Gathered 

Amount Info 
Colleeted 

Time to Reach 
Informed 
Deeision 

PRO/ 

CON 

1 

Portals using 
metrics 

Tl/2 

A 

<T1 

Best with 
little time 

2 

Portals using 
metrics 

T1 

A+B 

<t2 


3 

Portals using 
metrics 

T2 

A+B+C+D 

<T4 (=t3) 

Best with 
more time 

4 

Portals w/no 
metrics 

T1 

A 

T2 

Worst 
with little 
time 

5 

Portals w/no 
metrics 

T2 

A+B 

T4 



Table 5, lEEM Analysis Impact on Time to Reach Decision 


There are many variables that eould skew the results for time to reaeh a business 
deeision to be consistent (i.e., individual skill sets and experienee ean vary greatly). 
However, if improvements are made in the other metric areas outlined by the lEEM in 
Chapter III, Conceptual Framework, they will collectively help to minimize the time to 
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locate desired information and the resulting deeision reaehed. Time to locate is a elassie 
example that helps put the figure and table above into perspeetive. Oeeasionally, users 
may find what they are looking for sooner, but will also eontinue to look for long periods 
of time (perhaps as mueh as the approximate 50% of their time as they do now) because 
they are finding more of what they are seeking. Regardless, the time faetor is redueed 
with respeet to finding what is sought or eonsidered desirable: If people still spend 50% 
of their time looking for information, they should have more pertinent information than 
before in the same amount of time - whieh should lead to better deeision making and 
ultimately more effeetiveness. Time to loeate is an ejjiciency metrie that is affeeted by 
inereases in effeetiveness elsewhere. The effieieney metrie of time to loeate in turn 
affects effectiveness across the board beeause users will either have or ean do more in 
less time. Thus, this metrie area is an example of how effectiveness affeets effieieney 
and then how this effieieney inerease in turn improves effeetiveness. 

F, DERIVING AND EMPLOYING A COMMON UNIT OF ANALYSIS 

Time is money and the unit of analysis most appropriate to measure the ROI 
impaet of the lEEM metries based analyties is indeed time. Although it may be 
ehallenging to put exact figures on the impaet of every intranet portal project, placing a 
cost-benefit ROI on the worth of applying the lEEM and metrics baseline analyties aeross 
a large enterprise is attainable. The key assumption here is that a eorporation would 
assume the lEEM’s supporting metries to serve as a baseline to perform analyties only 
when it already has an intranet (i.e., sunk eost as part of doing business) and it believes 
there is room for eontinuous optimization in it to inerease value. The ROI eondueted is 
essentially a eost-benefit between the time needed to invest in applying and aeting upon 
the results of the analyties (i.e., eosts to hire an outside analyst or establish an in-house 
team to eonduet the analyses) and the results in time reduetions due to subsequent 
ehanges introduced by this analysis proeess, i.e., shorter completion time of a series of 
business related tasks (sub-eorporate level) and speed to market or eompletion of a 
project (corporate level). The inereases in quality, ereativity and knowledge worker or 
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customer satisfaction will be strongly implicit but each falls short of readily breaking 
down into precise units of time, only approximations of it. 

Time savings is sought through the optimization of intranet portals which are 
directly correlated to impacts on both efficiency and effectiveness gains in support of 
business value objectives. As outlined in Section V.E, Objective of ROIMI, this can be 
accomplished only if the same groups of metrics are applied periodically to obtain before 
and after results. Otherwise, the comparison between the two sets of data collapses and 
taking subsequent actions to enhance performance related to the results of specifically 
crafted groupings of metrics are less certain and valid. The reductions in time must be 
compared against previous baseline measurements to gauge the extent of performance 
improvements (see Section IV.C: Periodic Review and Figure 10; Implementation 
Scenarios for Applying a Baseline Metrics). 

Practitioners who redesign business processes require a method for determining 
how much their process design decisions will impact performance (El Sawy, 2001). 
During the lifetime of this approach other combinations of metrics may be applied which 
are deemed more precise, but these should only take place after at least two to three 
periodic measurements have already been fully conducted and analyzed to mitigate 
anomalies and correct errors. This method thereby provides a convenient way to estimate 
the returns that alternative process design changes can generate. Thus, the lEEM 
framework portends to resolve the long-standing problem in the IT community of 
determining the IT initiative impacts on a large number of processes at precise enough 
levels of the entire find experience to benefit managers who must implement changes at 
the tactical level and still link them to strategic business objectives. The rest of this 
chapter explains how. 
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1, Entropy Concept 


The credibility and applicability of this conjecture are significantly fortified by 
associating the metric performance indicators to legitimate and logical granular unit of 
measurement. The technique which meets the requirement of determining the output of 
time savings as well as enabling its “operationalization” of this theory is the Knowledge 
Value Added (KVA) theory which offers a practical method for estimating the value 
added by IT via theories rooted in assumptions derived from entropy in complexity and 
thermodynamics concepts: 

The changes organizational processes make in the structure of inputs to 
produce outputs can be described in a common way in terms of the 
entropy concept. The concept of entropy is defined as a measure of the 
degree of disorder or change in a system. In the context of business 
processes it can be used as a surrogate for the amount of changes that a 
process makes to inputs to produce attendant outputs. These process- 
induced changes can be measured in terms of the equivalent corresponding 
changes in entropy (Housel and Kanevsky, 1995). 

Within the framework of thermodynamics, a fundamental parallelism between 
transformation of substances and information processing has been established (Li and 
Vitanyi, 1993). If a substance is transformed from state a to state b, then the difference 
of the entropies, i.e., AE=E(b)-E(a), is proportional to the amount of thermodynamic 
work required for the change (Housel et al, 2001, p. 11). In other words, application of 
knowledge is determinant of value. A process must enact some change upon inputs to 
produce an output of value. Therefore, change can create value and knowledge is 
proportional to value. 


As theorized by Housel, Rodgers, El Sawy and Zhong, by extending this 
conceptualization of the relationship between complexity and entropy in 
the organizational context, conditional complexity can be viewed as the 
shortest description of the process, i.e., effectively, the productivity of the 
process. Further, a change in entropy when state a is transformed into 
state b depends only on a and b and does not depend on process P. This 
means that, by definition, any process P that changes a into b introduces 
the same change in entropy or, in an organizational context, adds the 
same value. It is reasonable then to assume that the minimal set of 
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instructions to change a into b, via process P reflects the corresponding 
change in entropy given the current state of process P. In other words, 
the length of the shortest description of the change provides an aeceptable 
approximation of the change in entropy given the eurrent state of the 
process. This becomes eritical in reeognizing that estimations of ehanges 
in entropy can only ever be approximations (Housel et al.., 2001, pp. 12- 
14). 

Thus, given that the estimates are derived using the common theoretical framework in 
lEEM, it follows that a simple correlation between process and outcome leads to 
reasonable approximation of the reliability of the estimates. 



At a given point in time, a company’s total process outputs produee its revenue. 
It follows that the proeedural knowledge required to produce those outputs is a surrogate 
for the revenue. Eurther, if this proeedural knowledge, which is distributed among people 
and IT, can be deseribed in common units, then it is possible to allocate corporate 
revenue to these units of knowledge. This would allow establishment of a common price 
per unit of procedural knowledge. It follows that price per unit of procedural knowledge 
is a surrogate for price per unit of common output. This formulation allows a direct 

® Housel et al., 2001, p. 14. 
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linkage between eorporate revenue and the proeedural knowledge distributed among the 
people and IT used to produee the revenue. Hence it would be possible to allocate the 
proportionate revenue produced by the procedural knowledge in business processes 
including the knowledge contained in the supporting IT. This approach establishes the 
relationship between cost and resulting productivity (Housel et ah, 2001, p. 13). 

2. Surrogate for Value 

The relationship between change in entropy and value added, while fundamental, 
does not provide a practical way to calculate the value-added by organizational processes, 
i.e., the entropy increment (Housel et ah, 2001, p. 13). The time it takes the average 
learner to acquire the procedural knowledge required to produce a process output 
provides one practical surrogate for the corresponding changes in entropy (see Figure 13, 
Procedural Knowledge is Proportionate to Change). This framework can be applied to 
the context of organizational processes: Processes with predetermined outputs may be 
described in terms of the amount of time it takes the average knowledge worker to 
produce those outputs they normally do to complete a business task. It follows, that the 
procedural knowledge used to produce the attendant outputs may be viewed as a 
surrogate for the process outputs. 

The impact of procedural knowledge can be viewed in two respects in the context 
of lEEM and portal optimization: one is the reduction of knowledge worker task 
completion time to exercise the same level of procedural knowledge to create equally if 
not improved outputs (this respect is how KVA can be applied and is analogous to 
learning time to conduct a task that has been assigned a percentage of revenue generation 
for a known outcome whose revenue returned can be estimated); and other is in the 
context of cost benefit showing how more tasks (ergo more expected outcome) can be 
accomplished in the same period of time or, alternatively, how the same expected 
outcome can be achieved in less time (this respect is how Activity Based Costing can be 
applied, see Section E.2.b: Activity Based Costing and lEEM). 
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In the knowledge worker task eompletion time approach, the total amount of task 
completing time required to business-related outputs (i.e., research, code or service) is a 
surrogate for the revenue derived from a firm’s outputs during a given sample period. 
The outputs of all the company processes used to generate this revenue, at a given point 
in time, can be described in common units of task completion time. It follows that “price 
per unit of output,” or its surrogate “price per unit of knowledge,” (which is derived by 
dividing company revenue by the total number of units of knowledge) is a constant. 
However, the cost per unit of knowledge will vary depending on the cost of the 
knowledge resources (e.g., people and technology) used to produce a process output 
(Housel et ah, 2001, p. 16). 

One task for example in which the lEEM can be calculated with KVA is through 
greater awareness of content which allows for greater opportunities for such things as 
software reuse or the reduced time to complete any standard business routine. Even more 
important than reuse or number of lines of reduced code per se is the derived benefits 
portals provide in improving the chances for faster development and time to market. Eor 
example, many additional products are produced as a result of discovering code or useful 
information that would have been otherwise inconceivable. Thus, the usability of 
valuable documents and artifacts previously unrecognized creates competitive advantage. 
Since the over-arching objective of an intranet is time reduction to complete all tasks, 
enhancements to process design as a result of lEEM analysis to optimizing portals for 
knowledge worker productivity needs is therefore the most crucial issue in facilitating 
and maintaining the highest returns possible for an intranet. The advantages of 
employing the Knowledge Value Added and Activity Based Costing methodologies in 
conjunction with lEEM are that, while grounded in a theoretical framework, they can be 
applied practically to obtain estimates grounded in common units and that these units 
can be used as a surrogate for value as well as compared to each other to ensure 
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opportunities to increase value are not overlooked. How these two approaches to 
estimating ROI compliment each other are addressed in more detail in the remainder of 
this section. 


a. Knowledge Value Added and lEEM 

As changes in process design may be the most crucial issue in predicting 
and maintaining the highest ROIMI to best leverage knowledge embedded practices 
within and across intranet portals, good old-fashioned ROI modeling does not permit 
enough time to develop the business case for either the metrics or the changes suggested 
by their results. Therefore, calculating returns using the KVA approach in conjunction 
with the lEEM works like an investment-portfolio approach: the changes made to process 
are thought to payoff and improve value, but a period of time is needed to collect the data 
periodically and analyze it before value realizations can occur, generally in the form of 
recognizing greater savings in time or, conversely, more productivity-related activity in 
less time as a result of the application of and enhancements to the exercise of procedural 
knowledge. 


The essence of KVA is that it takes knowledge utilized in corporate core 
processes and translates it into a numerical form that allows allocation of revenue in 
proportion to the value-added by the knowledge as well as the cost to use that knowledge 
(Baskerville, 1999, pp. 20-21). Tracking the conversion of knowledge into value, while 
measuring its bottom line impacts, enables managers to increase the productivity of these 
critical assets — namely in this study, the crucial process activities that take place in 
portals that drive productivity. 

Although the KVA methodology can be applied at any level in a company, 
it takes on significant value when applied at the enterprise level. A form of the KVA 
methodology when used within the lEEM framework allows a business intelligence team 
to iteratively generate estimates of return on portal information system related 
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initiative/process changes as they test/tweak various proeess designs modifieations within 
and aeross a variety of portals. In this manner, eompetitive advantages of this faeulty, 
i.e., ehanges brought about by the induetion and deduetion of metrie results based off the 
lEEM, ean be refleeted in eontributions to the eompany bottom line over time, not 
overnight. 


ROIMI essentially boils down to a delta in time savings as the time 
devoted to applying, analyzing and taking subsequent aetions based on the lEEM metrie 
eonversion rates results in a net gain in time — or what ean be aeeomplished in that same 
period of time — in the work of all workers within the enterprise. By extension, this gain 
in time can be plausibly extrapolated into a gain in value (see Appendix D; Time to 
Locate, for explanation of how this is taken into aeeount by the lEEM). One high level 
method of expressing the time in terms of savings, whieh is easier and more justifiable 
for some parties as opposed to value ereated, ean be done by representing the delta in the 
time devoted by the small group of people responsible for making all the proeess ehanges 
to portals eompared to aggregate statisties at eaeh portal previously and then to the 
enterprise as a whole prior to the ehanges taking affeet. 

Eor example if Time gained (zl7) as a result of time invested into applying 
and aeting on lEEM metries, then AT equals the amount of time gained by all employees 
from one measured period of time to the next A{eT)= eTi-eT 2 minus the total time spent 
by the business intelligenee team {bitT) in applying and analyzing a baseline of metries 
uniformly aeross the enterprise, effeetively affeeting a wide bodies of knowledge workers 
(as well as others eausality speaking). A simple expression of this is zir= A{eT) - (bitT). 
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An example of how this equation may play out in a large eorporation is: 

AT = A{eT) - (bitT) 

A{eT) = eTj - eTi 

First 30 Period Second 30 Period 

Prior- metric changes Post- metric changes 
eT I — Total No. K-Workers x Total Hours On-Line 

5000 X 4 hr/day x 20 days = 400,000 hours 

eJj = Total No. K-Workers x Total Hours On-Line 
5,000 X 3.8 hr/day x 20 days = 380,000 hours 

A{eT) = 400,000 - 380,000 hours 
= 20,000 hours 

AT = A{eT) - (bitT) 

20,000 hours 

(bitT) — Total No. in BIT x Total Hours Worked 
10 X 4hrs/day x 20 days 
= 800 hours 

AT = 20,000 hours - 800 hours 

= 19,200 hours gained during second period 


In this example, there is a 1:24 ratio in time invested by BIT and time 
saved to other knowledge worker company wide as a result of their process changes. 
This assumes that all portals are being measured and that most employees use their 
intranet to conduct business. Although this ratio difference increases with the size of an 
organization and its intranet, it will, conversely, diminish at some point and go the other 
directions as these employee numbers and intranet sizes decrease. Even if one quarter of 
the estimate above is realized during a 30 day period, this enhancement to the exercise of 
procedural knowledge would still result in a theoretical saving of over 57,000 hours per 
year (.25 x 19,200 x 12months). It is these kinds of numbers that will get attention and 
prove why focusing on changes to process is the key to higher returns. 


These results can be expressed as magnitude percentage estimates, 
something mangers at multiple levels may be more willing to share and allow further 
interventions since specific dollar amounts resulting from the interventions may be 



hidden from competitors while being able to share the general results of these 
interventions. Another tactic in presenting performance metrics and ROI in this case is 
that it may be wiser to use a range of numbers rather than a single target point. Low-end 
numbers enable management to make a decision based on conservative projections. 
High-end numbers dangle the prospect of bigger potential gains in front of them, 
informing them in any event of the significant scale these changes can make (see 
Appendix F: Spreadsheet of ROI Returns for examples of both ROI magnitude 
percentage estimates and target range of numbers). A clearer understanding of the 
potential return should encouraging BDMs to take risks (with such investments of time 
invested by a few knowledge workers to reduce time on task for many knowledge 
workers) until they have an accurate way of gauging potential value-creating benefits in 
harder numbers. In summary, this method provides a means - as well as a rational 
justification — in which to calculate ROI for the metrics investment (time saved and 
better used by enhancements to procedural knowledge within processes) with a common 
unit of analysis: time to exercise procedural knowledge to produce an expected outcome, 
which can always be translated into money saved or earned as well as anecdotally 
accepted as boosting competitive advantage, i.e., time to market. 

This example also helps to explain why a form of the KVA methodology 
can be made to be applicable. For one its basis in entropy concepts takes into account 
change or the process that enables this and that these changes/outputs are the thing of 
value since customers pay for the output of the corporation at any given point in time. 
Simply put, it allows allocation of revenue to the corporate and sub-corporate levels and 
it allows description of all process outputs in common units. These common units are 
proportionate to revenue and thus revenue can be allocated to these units allowing the 
generation of a numerator for an ROI ratio that is not based on any form of cost 
including, cost savings, cost avoidance, investment cost, etc.^ 


^ Housel, email to Grant Jacoby on 19 November 2003. 
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In the case of the example above, KVA could be applied to the lEEM for 
the time it takes someone to do their job without optimizing changes being made to the 
process compared to someone who has the same tasks but with the optimized settings. It 
would be something akin to learning or doing a job and not having to learn or do as much 
to in order to accomplish the same requirement as a result of the application of 
knowledge exercised by the analysis team in the form of the changes they make to crucial 
productivity processes of portal intranets. 

In other words, the time to exercise knowledge based off the metrics and 
applied to impactful productivity process changes of portals can sow even greater 
knowledge gains into the enterprise at large, i.e., the 1:24 time savings ratio given above. 
With regard to efficiency and effectiveness, time is saved and value created by virtue of 
people being able to do more in less time — again, be it learning or doing. 
Concomitantly, since these process changes ultimately go across multiple portals there is 
an effectiveness gain as well, creating value. 

b. Activity Based Costing and lEEM 

Another and complimenting technique that can be used to measure ROIMI 
for lEEM based metrics is Activity Based Costing. ABC is a popular cost-based 
approach because finding the true costs of process activities are clearly useful in 
evaluating them (Johnson and Kaplan 1987). Despite the fact that the ABC technique 
appears to be a very suitable managerial tool for e-business, widely-known published 
reports about its use in intranet or internet-related economies do not currently exist. 
Nevertheless, applications of ABC to measuring the impacts of IT assume that any 
cost/time saved or processes simplified (and thus costs or time reduced) by the IT 
(namely in this case the corrective actions taken after analysis of lEEM metrics and 
conversion rates) are a direct reflection of its value. This assumption holds true in cases 
where lEEM metrics analysis is applied causing reductions in cost/time while process 
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outputs remain constant or increase. Thus, the applicability and merits of ABC to lEEM 
related ROIMI warrants further examination. 

Criticisms of ABC need to be kept in mind and overcome if it is to have 
credibility in assigning value to process changes brought about by analysis of metrics. 
The conceptual limitation of the cost-based approaches to generating a return on 
investment-type performance ratio is that they do not have a surrogate for revenue 
(Johnson 1992). The problem of using this method for evaluating the value added by IT, 
is the fact that if cost (or any of its derivatives) is used as a surrogate for value, then all 
the information is contained in one term of the ratio, i.e., the denominator (Housel et ah, 
2001, p. 10). The data source for value should come from the revenue side of the firm's 
performance (i.e., numerator) and the data source for cost (i.e., cost) should come from 
the cost to produce the firm's outputs. In the case of lEEM, a form of ABC provides a 
numerator of procedures to accomplish an activity over the time and cost it takes. This is 
akin to and borrows from the KVA methodology and presents a method to measure and 
trace value at the sub-corporate level, ironically unlike ABC’s originally intended design 
(see Section II.C.2: Cost-Based Similarities and Differences and Table 1, Common 
Approaches to Measuring the Return on IT). Nevertheless, conventional application of 
ABC is strictly about cost. This approach to ROI can compliment that of KVA and an 
illustration of this follows. 

ABC is a systematic, cause & effect method of assigning the cost of 
activities to products, services and customers {cost objects). ABC uses a simple 
principle: 

• Products, Services and Customers generate the need for activities. 

• Activities consume resources. 

• The more varied and diverse the Products, Services and Customers, the more 
activities are generated and the more resources are needed. 
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ABC measures the eost and eonsumption of aetivities and assigns these eosts only to the 
eost objeet generating the aetivity, sueh as the serviee provided or the demand of a 
eustomer (Roztoeki, 2001, p. 2). ABC introduees the eoneept of cost drivers, wtiioli are 
any faetor that eause a ehange in the level of aetivity. In the ease of lEEM metries 
analysis and notions, it is the prooess ohanges, speoifioally the prooedures removed, 
modified or introduoed. It is the ohoioe and use of eost drivers that enables the analysis 
team to aoourately allooate the indireot and overhead eosts to the appropriate eost objeet. 
Eor example, assigning resouroe eosts assooiated with looking and prooessing 
information to do a job (aetivity) to provide a serviee (eost objeet) oan be aooomplished 
by using the number of searohes, navigational steps, oliokstreams, and other metries 
outlined by the lEEM metries baseline and eonversion ratios (eost driver). The better the 
serviee (be it faster or a qualifiable improvement), the less time (resouree) is used to run 
through the steps neeessary to provide the serviee, the less eosts are assigned to this 
serviee. By foeusing on the minimization or optimization of an aetivity by either 
redueing the number of prooedures or strengthening them to aooomplish an aetivity 
through oruoial prooess ohanges of intranet praotioes, the number of prooedures reduced 
provides a means to measure eonsumption of resouroes as well as a traoe to where value 
is gained. In addition, the traoe on value further informs future deoisions regarding where 
prooess ohanges have the greatest impaot. 

Expenses whioh oan be assooiated with a partioular eost objeet are 
oonsidered “direof’, i.e., salaries and expenses and those whioh oan not be assooiated 
with a partioular eost objeet are defined as “overhead”, i.e., operational eosts. It is these 
eosts that oan be traoed from aetivities to eost objeots. To systematioally relate aetivities 
to eost objeots, the direot and overhead eosts of eaoh eost objeet are added together as 
“indireot” eosts in order to obtain the produot eost. The produot eost represents an 
estimate of the actual expenditure on the part of a oompany to generate a eost objeet, 
rather than the eost of that objeet to a eustomer. The remaining paragraphs in this seotion 
provide an example of how this would work in relation to lEEM. 
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An example of ABC derived ROIMI for lEEM ean be illustrated by a 
division that runs two sets of proeedures: one in a non-optimized portal and the other in 
an optimized portal essentially using a similar proeess but with less required proeedures. 
Stepping through any proeedures to do nearly any aetivity takes time and resouree. 
Additionally, proeess ehanges that result from aetions taken from metries analysis relate 
direetly to proeedures taken and take up time and resourees as well whieh need to be 
taken into aeeount when seeking a ROE 

Eor instanee, during a 30 day period to aeeomplish an aetivity, the direet 
and overhead eost assigned to a seetion is $2,400. An optimized portal’s eost would need 
to aeeount for the eosts devoted in optimizing the portal for that given period. Eor 
instanee, the eosts eould amount to $800 and this eost eould be aeeounted for during the 
period of just one aetivity or amortized over a longer period. It would be more realistie 
however to amortize these eosts over the eourse of at least one year over the same 
aetivities that take advantage of the same proeedure ehanges made to faeilitate the 
eompletion of an aetivity as a part of doing business, i.e., “time to value”. 

This aetivity example would inelude the following: 

• 310 proeedures are required to perform an aetivity in a portal not optimized 
(links, design, help). 

• 285 proeedures are required to perform the same aetivity in an optimized 

portal. 

• Proeedures in a non-optimized portal require 125 hours of work to be 
eompleted. 

• Proeedures in an optimized portal require 110 hours of work to be eompleted. 

• The additional eost assoeiated to the optimized portal for this aetivity is $800. 
In total, there are 310 proeedures to eomplete an aetivity in 125 hours that would 
normally eost $2,400 in one non-optimized portal versus 285 in 110 hours for an 
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optimized portal which requires an addition $800 to cover the optimization costs (albeit 
for one activity or amortized over the course of a year); 


To develop this example further, the additional costs imposed by factoring 
in the costs of metrics analysis and changes are factored two ways in the two 
complimentary tables below for comparative purposes: in one 30 day activity and then 
amortized over the course of a year for the same activity conducted multiple times by one 
knowledge worker. 


Indirect cost (time) assigned procedures in non-optimized portal - 
(310/125 hours) 

1 procedure per 
every 24 min. 12 sec 

Indirect cost (time) assigned procedures in optimized portal - 
(285 /no hours) 

1 procedure per 
every 23 min. 10 sec 

Average indirect cost assigned non-optimized portal over time 
period to complete the activity (285 / $2,400) 

1 procedure per 
every $7.74 

Average indirect cost assigned optimized portal over time period to 
complete activity (285/ $2,400 x 1 lOhours / 310 hours + $800) 

1 procedure per 
every $10.21 

Average indirect cost assigned optimized portal amortized over one 
year time period to complete the activity 
(285/ $2,400 X 1 lOhours / 125 hours + $800/12) 

1 procedure per 
every $7.64 


Table 6, ABC Costs Assigned to Portals 
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Variable Calculated 

Best Case 

Worst Case 

Difference 

Number of Procedures 




No. procedures required to perform activity in a non-optimized portal 

310 

310 


No. procedures required to perform activity in an optimized portal 

285 

300 

-15 

Number of Hours 




No. hours required to complete activity in a non-optimized portal 

125 

125 


No. hours required to complete activity in an optimized portal 

no 

116.5 

-6.5 

Number of Knowledge Workers that Conduct same Task 

500 

450 

-50 

Cost of Activity (direct and Overhead) 

2400 

2400 


Additional Cost (Associated to the optimized portal for this activity) 

800 

1100 

-300 

Indirect Costs for Non-Optimized Portal 




Indirect cost (time in minutes) assigned procedures in non-optimized portal 

24.19354839 

24.19354839 


Average cost assigned non-optimized portal over period to complete activity 

7.741935484 

7.741935484 


Indirect Costs for Optimized Portal 




Indirect cost (time in minutes) assigned procedures in optimized portal 

23.15789474 

23.3 

-8.526315789 

Average cost assigned optimized portal over time to complete activity 

10.21754386 

11.12266667 

-0.905122807 

Amortized Indirect Costs for Optimized Portal 




Average cost assigned optimized portal amortized over one year period 

7.644444444 

7.761555556 

0.117111111 

Frequency of Activity (answer only one choice below) 

12 

12 


Weekly (enter the value of 1 if this applies) 




Bi-Weekly (enter the value of 1 if this applies) 




Monthly (enter the value of 1 if this applies) 

1 

1 


Bi-Monthly (enter the value of 1 if this applies) 




Quarterly (enter the value of 1 if this applies) 




Semi-Annually (enter the value of 1 if this applies) 




Annually (enter the value of 1 if this applies) 




Procedures Gained in One Activity Period 

38.86363636 

21.88841202 

-16.97522435 

Percentage Procedures Gained in One Activity Period 

4.472140762 

3.834971618 

-0.637169144 

Procedures Gained in One Year for this Activity 

166.3636364 

142.6609442 

-23.70269216 

Percentage Change in Procedure Productivity for Activity in 1 Year (%) 

4.472140762 

3.834971618 

-0.637169144 

Total Cost of Procedures for this Activity in One Year 

29709.09091 

29980.25751 

271.1666016 

Savings of Procedures for Activity in Optimized Portal over One Year 

378.8856305 

-75.78568462 

-378.8856305 

Percentage Savings of Procedures in Optimized Portal over One Year 

1.280019022 

-0.253463828 

-1.53348285 

Annual Savings to Enterprise for Activity (if more than one employee) 

189442.82 

-34103.5581 

-223546.3733 

Percentaae Annual Savinas to EntepBrise for Activits 

1.315502 

-0.26312241 

-1.578624428 

Additional Similar Activities Accomplished across Enterprise 

291.8660287 

213.9914163 

-77.8746124 

Percentage Additional Similar Activities Accomplished across Enterprise 

4.864433812 

3.962804006 

-0.901629806 

Virtual K-Workers Gained for Activity across Enterprise 

24.32216906 

17.83261803 

-6.489551033 

Percentage Virtual K-Workers Gained for Activity across Enterprise 

4.864433812 

3.962804006 

-0.901629806 

Table 7, ABC ROI Returns for 30 Day Activities 

During One 

Year 
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If knowledge workers in the optimized portal worked the same number of 
hours as the non-optimized portal, they would be able aceomplish approximately 39 more 
proeedures within the same time period (310 hours / 612 seconds per procedure). Taken 
a step further if this activity is done 12 time in the course of a year (125 hours is 
approximately half the number of hours one person works a month, therefore this activity 
would only account for half of their jobs) and the costs are tabulated using the 
amortization of the $800 development costs over that period, the total number of 
additional procedures accomplished would be 166 procedures ([38.86 procedures 
accomplished in optimized portal/ month x 12 months] - [24.19 procedures accomplished 
in non-optimized portal/ month x 12 months ) and the cost would be $29,709.09 (3886.36 
procedures / year x 12 months x $7.64 / 1 procedure). 

In summary, the additional 166 procedures gained would cost an 
additional $378.89 as opposed to $909.09*, providing a 4.47% increase in procedures 
accomplished during that month. Taken collectively across the enterprise the percentage 
increase (or ROI) would continue to gradually climb (albeit slightly in some activities 
and indistinguishable in others if the process is near fully optimized), moreover the cost 
savings would be profound. The example given is prepared in the context of what one 
knowledge worker can accomplish in the average amount of work hours per month. If 
the savings for this one knowledge worker is nearly $379.89, the savings would be even 
larger every year when applied to an even greater number of them, i.e., $379.89x 500 
workers who must accomplish this same activity equates to $189,442.82 annual savings 
for this one activity alone. Similarly, an additional 291 activities can be accomplished 
(166 procedures / year x 500 k-workers = 83,181 procedures / 285 procedures per 
activity accomplished in an optimized portal): the equivalent of adding 24 knowledge 
workers (291 activities / 12 activities per k-worker per year) or, alternatively, providing a 
justification to reduce the size of a knowledge worker pool who completes this activity. 


(Total Cost of all procedures completed in optimized portal) versus (Total Cost of procedures in non- 
optimized portal had it had to produce the same number of procedures as the optimized portal in the same 
period of time) -($800 Total Cost of activity x 12 months) 

96 




Although the percentage improvement appears nearly insignificant for the 
first activities, the impact of being able to perform more procedures with the same cost or 
less procedures to accomplish an activity in less time is significant when applied across 
the enterprise for all knowledge workers (or groups) who perform this same activity over 
time, as Figure Productivity Pyramid below illustrates in purple (color for efficacy gains) 
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This type of calculating is conceivable and doable for standardized 
processes of work that can be enhanced by portal changes in information access and 
discovery. However, it is limited in accounting for creative processes since the steps of 
each procedure cannot be known with certainty ahead of time. It can provide 
approximate accountability by gauging levels of disintermediation achieved through 
conversion rates, anecdotal confirmations by the knowledge workers through surveys and 
more general time estimates derived from shorter development times achieved by in- 
house research and development efforts or cycles within the R&D department itself. 

The example of procedures used above can take on other parallel meanings 
in terms of value that can be derived. For example, if computer code, i.e., instruction sets 
(another surrogate for procedures), is discovered as a result of it being more accessible, 
there would also be significant savings. Using the information provided by ABC as a 
means to monitor impact of process changes (as a result of lEEM metrics analysis), 
companies in effect can cut costs, identify opportunities for improvement, and determine 
a more profitable way of conducting business activities. In addition, the output of the 
ABC analysis is a good basis for revising tactical-level portal changes/enhancements as 
well as efficiencies expected of corporate portal strategies (see Appendix E: Spreadsheet 
of ABC ROI Return Examples to see a variety of different activities conducted at varying 
intervals each year, ie., weekly, bi-weekly, quarterly, etc.). Appendix E also provides 
worst and best case estimates of process changes to provide BDMs a measure of risk 
involved with each series of process changes. Eor example, in the Quarterly Activity 
estimates, the best case is an annual savings gain of $428,571 but poses a risk of -$35,200 
in the worst case. Given this range, the BDMs can better ascertain if the changes are 
worth the risk. 
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ONE YEAR RESULTS 

Variable Calculated 

Number of Procedures 

No. procedures required to perform activity in a non-optimized portal 
No. procedures required to perform activity in an optimized portal 

Number of Hours 

No. hours required to complete activity in a non-optimized portal 
No. hours required to complete activity in an optimized portal 

Number of Knowledge Workers that Conduct same Task 
Cost of Activity (direct and Overhead) 

Additional Cost (Associated to the optimized portal for this activity) 


Weekly Weekly Bi-Weekly Bi-Weekly Monthly Monthly Quarterly Quarterly 
Best Case Worst Case Best Case Worst Case Best Case Worst Case Best Case Worst Case 


50 

45 

30 

26 

4000 

700 

200 


50 

48 

30 

28.5 

3500 

700 

300 


no 

95 

68 

57 

1000 

1200 

400 


no 

102 

68 

64 

900 

1200 

700 


310 

285 

125 

no 

500 

2400 

800 


310 

300 

125 

116.5 

450 

2400 

1100 


1200 

1100 

400 

350 

250 

15000 

1000 


1200 

1150 

400 

375 

220 

15000 

1400 


Frequency of Activity (answer only 1 choice) 


48 


48 


24 


24 


12 


12 


Procedures Gained in One Activity Period 6.923 2.526 18.33 6.375 38.863 21.888 

Percentage Procedures Gained in One Activity Period 3.846 1.052 3.03 -1.477 4.472 3.834 

Procedures Gained in One Year for this Activity 92.30 25.26 80 -39 166.363 142.66 

Percentage Change in Procedure Productivity for Activity in One Year 3.846 1.052 3.03 -1.477 4.472 3.834 

Total Cost of Procedures for this Activity in One Year _ 33830.76 33915.78 29277.19 29543.75 29709.09 29980.25 


157.142 76.666 

4.761 2.222 

228.57 106.667 

4.762 2.222 

61142.85 61493.333 


Savings of Procedures for Activity in Optimized Portal over One Year 1061.538 

Percentage Savings of Procedures in Optimized Portal over One Year 3.140 

4246153 

Annual Savings to Enterprise for Activity (if more than one employee) 

Percentage Annual Savings to Enterprise for Activity 3.159 

Additional Similar Activities Accomplished across Enterprise 8205.128 

Percentage Additional Similar Activities Accomplished across Enterprise 4.273 

Virtual K-Workers Gained for Activity across Enterprise 170.94 

Percentage Virtual K-Workers Gained for Activity across Enterprise 4.273 

TWO YEAR RESULTS 

Frequency of Activity (answer only one choice below) 96 


37.89 

0.111 

132631.57 


0.112 

1842.105 

1.096 

38.37 

1.096 

96 


395.53 

1.354 


-1169.204 

-3.963 


378.885 

1.28 


-75.785 

-0.253 


1714.285 

2.81 


842.105 

3.508 

35.087 

3.5087 

48 


-344.117 

-1.593 

-14.338 

-1.593 

48 


291.866 

4.864 

24.322 

4.864 

24 


213.991 

3.9628 

17.832 

3.962 

24 


51.948 

5.1948 

12.987 

5.1948 


-160 

-0.26058 



20.4057 

2.31884 

5.101449 

2.31884 


Procedures Gained in One Activity Period 6.923 2.526 18.33 6.375 38.863 21.888 

Percentage Procedures Gained in One Activity Period 3.846 1.052 3.03 -1.477 4.472 3.834 

Procedures Gained in One Year for this Activity 184.61 50.52 160 -78 332.727 285.3218 

Percentage Change in Procedure Productivity for Activity in One Year 3.846 1.052 3.03 -1.477 4.472 3.834 

Total Cost of Procedures for this Activity in One Year _ 67430.76 67515.78 58077.19 58343.75 58509.09 58780.25 


157.142 76.667 

4.761 2.222 

457.142 213.333 

4.761 2.2222 

121142.8 121493.33 


Savings of Procedures for Activity in Optimized Portal over One Year 

Percentage Savings of Procedures in Optimized Portal over One Year 

2353.846 

3.492 

391.5789 

0.58 

1268.261 

2.186 

-1594.659 

-2.735 

1666.86 

2.8542 

1028.68 

1.752 

4571.428 

3.778 

1173.33 

0.966502 

Annual Savings to Enterprise for Activity (if more than one employee) 

Percentafiii Annual Savins to Enl^tlTise for Ai^dl^ 

9415384.62 

3.502 

1370526.31 

0.582 

1268261.56 

2.201 > 

<143519^2^ 

-2.76^ 

833431.08 

/ 2.893 

462908.76 

1.785 

1142857.1 

3.8093 

258133.33 

0.97772 

Additional Similar Activities Accomplished across Enterprise 

16410.25 

3684.210 

1684.210 

-688.235 

583.732 

427.982 

103.896 

40.8115 

Percentage Additional Similar Activities Accomplished across Enterprise 

4.273 

1.096 

3.508 

-1.593 

4.864 

3.962 

5.1948 

2.31884 

Virtual K-Workers Gained for Activity across Enterprise 

170.94 

38.37 

35.087 

-14.338 

24.322 

17.832 

12.987 

5.101449 

Percentage Virtual K-Workers Gained for Activity across Enterprise 

4.273 

1.096 

3.5087 

-1.593 

4.864 

3.962 

5.194 

2.31884 


Table 8, One Year vs. Two Year ROI Returns for Different Activities 
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Despite the promising benefits of this technique based on the lEEM 
baseline of metrics and conversion rates, it does require time to gauge and calculate, 
which runs counter-grain to the fast pace nature of internet economies. An illustration of 
this using the previous 30 day example is depicted in Tables 8 {One Year versus Two 
Year ROI Returns for Different Activities) which illustrates how changes in worst case 
estimates can go from a negative projection (-$34,103.55 or -0.263% ROI) after one year 
to a positive return after two years ($462,908.76 or 1.785% ROI). On the contrary, some 
investments may never - or for an unacceptably long period of time - provide a positive 
return under worst case estimates, such as the bi-weekly scenario in Table 8 during one 
and two year returns, providing negative 4.059% and 2.768% ROIs respectively. It is up 
to the BDMs to decide what is an acceptable risk, but they must first be given the 
expectations in terms they understand and which can be rationalized by a sound model 
supported by mathematics (see Appendix E: Spreadsheet of ROI Returns for more 
examples and an explanation of the calculations). Given the time it would take to realize 
value gains is affordable, ABC-like estimates of savings from lEEM metrics, i.e., semi¬ 
annual to annual results, appear to be a good managerial tool to gauge time to value for 
intranets (as well as internets) of large companies involved in e-business. 


3. Keys to Measuring Returns on IT 

Although they are inter-related, time is the efficiency factor and creating value is 
the effectiveness factor. A tractable method to prove this with any hard numbers would 
be similar to the examples above in the form of time reductions in the exercise of 
procedural knowledge — much like KVA does in the form of return on knowledge in case 
example it uses showing differences of learning times before and after the application of 
knowledge (Housel et al. 1999). Consequently, KVA is proposed in this paper as one 
way to estimate the value-revenue allocatable to corporate assets such as people and 
technology. And ABC is proposed as another means to estimating the return or cost- 
benefit of ROIMI in a tractable procedure presented in terms of value gain, be it cost, 
time, activities generated or number of knowledge workers required. In both 
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methodologies, ereating value ean be eonveyed through the inerease of conversion ratios 
(found in the metrics sub-grouping of Appendix D) that constitute critical business 
requirements, such as loyalty, reach and disintermediation. 

KVA and ABC methods of estimating ROI also compliment each other when 
used together to estimate the same process. For example, audits that result due to 
discrepancies can be automated quickly while others are more manual intensive and 
require time to resolve. In any event, significant cost savings can be made if the number 
of discrepancies that require audits is lowered. In one study, KVA analysis does not 
make any recommendations for changes in auditing function because on paper it has 
relatively high ROK (cost to learn how to conduct audits divided into the revenue created 
by them when factoring associated percentage of costs of audits and the revenue 
generated back). From an ABC perspective, however, the auditing function comes under 
scrutiny because of the high cost when a discrepancy has to be researched. Intuitively, 
auditing does not add value for the customer and, therefore, is a target for re-engineering 
discovered by ABC (Nomura, 2002). Thus, when using KVA, an analyst must be careful 
to factor in qualitative measures (i.e., common sense) to ensure a thorough and complete 
re-engineering effort is made. 

On the other hand, ABC has deficiencies when dealing with processes that are 
complex or involve a large amount of knowledge. In such processes, costs and ROK will 
not be correlated and, consequently, re-engineering efforts will be focused in different 
areas. As a result, ABC may misdirect re-engineering efforts. As the economy shifts 
from a manufacturing to a services emphasis, the value of the KVA methodology 
increases. Knowledge intensive processes are more prevalent in the services sector and, 
therefore, will benefit the most from a re-engineering project using the KVA 
methodology. Thus, while ABC is useful, due to our ascent into the "Information Age", 
KVA appears to be more relevant for the future (Nomura, 2002). 
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The advantage of eombining lEEM and its assoeiated baseline of metrics which 
emphasizes surveys and includes the key conversion ratios to estimate the improvement 
of critical business requirements with ROI estimates from both KVA and ABC is that 
they collectively overcome a limitation of the KVA and ABC techniques when applied 
alone, not readily representing to the same extent increases in quality, creativity and 
knowledge worker satisfaction. However, these factors do impact the bottom line and 
will eventually find their way into processes with predetermined outputs because the 
most intangible asset of employee knowledge eventually becomes a tangible asset 
embedded in company IT. Though it is unlikely that the benefits of these factors will 
ever be completely quantified, over time this approach does allow for eventual 
accountability of conversions such as creative outputs into value since they are inevitably 
embedded into processes with predetermined outputs (Housel et al, 2001). Taken as a 
whole, conclusions from these approaches should be plausible and provide management 
with a more comprehensive picture of the value and direction of their intranet 
refinements and initiatives than they currently get from any other means or methodology. 

As the academic community points out, there are four key issues that need to be 
addressed within any framework for measuring the return on IT. 

• Unambiguous allocation of value as well as cost of IT initiatives 

• Mapping of IT economic impacts at any level of aggregation 

• Common unit of measurement 

• A supporting theoretical framework 

Together the lEEM and its supporting metric conversion ratios and their analysis along 
with the application of a form of KVA and ABC to determine ROI in measurable 
common units meet all four of these essential requirements. The collective framework 
and approach is theoretically-based and “operationalizable”. Eurther, such a framework 
can prove useful to the practitioners who are struggling to determine which IT process 
designs will provide the best returns from their intranet portals. In effect, consistent 
application of the lEEM framework and baseline metrics in combination with KVA and 
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ABC improves insight into how to increase the value of an enterprise in a rational fashion 
using common units of measurement when and where necessary. 


G. SUMMARY 

ROI is one of those things that, in theory, makes perfect sense. The problem with 
relying solely upon financial techniques such as Net Present Value is that they don't 
necessarily capture all of the business benefits of an IT investment, nor do they help to 
evaluate all of the options available (see Appendix E, Common Approaches to IT ROI). 
Nonetheless, a rational and comprehensive pursuit of ROI can lead to the discovery and 
optimization of proper metrics that can both demonstrate the business value of intranets 
portals as well as guide efforts toward enhancements to them that will have the greatest 
ROI. 


Although costs vary widely, executives and knowledge workers down the line 
uniformly expect big benefits from intranet portals. Ironically, however, firms will spend 
big money but won’t measure the results. Sixty-one percent of firms don’t have any 
metrics to prove portal benefits (Gillet, 2001, p. 7). Reasons for the lack of measurement 
range from not knowing how to do this to taking the easiest, albeit not very insightful, 
course of action to collect simple metrics such as page hits. Assessing soft and derived 
benefits for intranets — such as improved customer service, satisfaction, collaboration, 
loyalty and quicker time to market — can be one of the most challenging tasks in 
determining ROI for intranet portals. Thus, by applying the lEEM metrics baseline, 
conversion ratios and analysis, an azimuth indicator showing how well a corporation is 
reaching and supporting its strategic business requirements is possible, provided a 
reasonable return on intranet metrics investment (ROIMI) of the costs of the analysis 
process is compared to the time benefits using a form of KVA and ABC. Exactly who 
should be designated to ensure metric results and analysis produce reasonable estimates 
of ROIMI is addressed next in Chapter VI, Business Intelligence Team. 
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VI. BUSINESS INTELLIGENCE TEAM 


IT, in particular, needs to be managed as a capital investment 

opportunity, rather than being merely a money pit of expense. 

John Berry, IT Consultant and Columnist 

A, ROLE OF BUSINESS INTELLIGENCE TEAM 

Successful IT projects and ROI require eollaboration and eommitment between 
business managers and IT professionals. Beeause analytie skills are extremely scarce, 
enterprises cannot afford to scatter these experts. Therefore, enterprises should ereate a 
Business Intelligenee Team (BIT), in whieh analysts and IT experts work closely together 
to support business managers in their deeision-making tasks. The BIT should be housed 
where it matters most - not too high disconneeted from the real world and not too low 
losing an over-arehing view. 

Creation of a BIT is important beeause IT professionals are increasingly 
involuntarily and voluntarily involved in the strategie and finaneial implications of IT. 
Involuntarily, more and more proposals for IT projects must include eeonomie and 
strategic justifications and, similarly, more and more software developers/developments 
are being pressured by customers and managers alike to justify the value of the software 
and IT enhaneements being delivered. Voluntarily, developers of new IT products and 
processes are now trying to promote them not only with operational justifications (faster, 
fewer errors, ete.) but with strategie and finaneial justifieations (“this new process will 
deliver more business value”). Although IT professionals are inereasingly involved, two- 
thirds of companies interviewed by one researeh group stated funding responsibility lay 
fully with IT — or IT had no role at all (Gillet, 2001, p.7). This doesn’t mateh up with the 
levels and types of responsibilities and expertise required to deduct and calculate IT 
related ROI. 

B, BIT COMPOSITION 

Sueeess in applying Web analytics requires collaboration between analysts, 

portals managers and BDMs. Having the neeessary skills in these three fields to colleet, 
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interpret and aet on information quiekly is a eompetitive e-business differentiator. 
Preferably, the people involved in Web analytics should be skilled in more than one 
discipline. Web analysis is, first of all, an analytics exercise, not an IT project. 

1, Analyst 

An analyst must be capable of: 

• Exploring data and discovering patterns, meaningful relationships, and 
anomalies. 

• Working with the IT department to develop insight into how to identify data 
from the Web and other sources for a specific analysis. 

• Using a palette of techniques, ranging from simple data aggregation, via 
statistical analysis, to complex data mining. 

• Being fluent with analytics tools. 

• Distilling the relevant parts and producing sound recommendations, based on 
the right set of metrics. 

2, IT Professional 

An IT professional must be capable of: 

• Working with a business manager to define the right requirements. 

• Implementing the required changes in an efficient way. 

• Supporting analysts by advising on the efficient storage of Web site data, 
integration of data with other sources and how to keep an historic overview of this data. 

• Working with analysts to identify the necessary data for a specific analysis 
and make sure it is accessible and understandable. 
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3, Business Manager 

A business manager, responsible for a portal site, must be capable of: 

• Interpreting the results and creating decision alternatives, together with an 
analyst and IT professional/portal manager. 

• Tying analytic results to the corporate objectives to ensure that decisions 
support the enterprise's business requirements. 

• Initiating process changes based on changes in portal sites and to follow up on 
triggers or anomalies reported from the respective intranet portals. 



Business 
Skills 


Establish 
requirement 

Monitor 

Link to corporate \results 
strategy 


Figure 15, Complimenting Skill Sets and Tasks for BIT ^ 


C. BIT ADVANTAGES AND SHORTCOMINGS 

While portal managers tend to have a short-range view; the BDMs have a long- 
range perspective and while portal managers ask how and when; the BDMs ask what and 


^ Buytendijk, 2001, http ://www. gl. lit, edu/gartner2/research/97600/97608/97608 .html 
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why? The analyst meanwhile strives to figure out as many answers to these questions as 
possible. Creation of a multi-diseiplinary and perspeetive BIT produees more advantages 
than shortfalls, partieularly for large intranets, in guiding IT projeets and determining 
their ROT Advantages inelude: 

• One group to eentralize a variety of analytie tasks (see seetion IV.B.4, A 
Single Reporting Service). 

• BDMs are involved up front during ROI benefits analysis and ean be held 
more aeeountable along with IT for aehieving the predieted benefits. 

• Engenders business aeumen of teohnieal workers. 

• Ineorporates a better mix of eommon sense, professional judgment, 
quantitative modeling and strategie perspeetive. 

• More likely to ferret out benefits buried in other lines of business - due, in 
part, to their eolleetive multi-diseiplinary baekgrounds. 

• Separate software proposals into those that have potential for ROI and those 
that are simply the eost of doing business (i.e., know when ROI justifieation is warranted) 

• Provides greater validity to ROI analysis, resulting in a wider spread of 
aeeeptanee. 

Disadvantages tend to affeet smaller organizations and inelude: 

• The Web exaeerbates the advantages of seale: A large enterprise will get far 
more leverage from its Web investments than a small one (Casser, 2001, p.3). 

• Some assets used to eompute value-based measures are in the form of 
intelleetual eapital, whieh runs up against a faseinating set of issues in figuring out how 
to value this talent (Meyers, 1997, p. 47). The result is that value-based performanee 
metries make little sense for eompanies without signifieant hard assets. 

• Depending on their levels of experienee, some BITs may make the mistake of 
thinking that measurement of eausation is mueh more prevalent than it really is, 
partieularly if too many modifieations are made to portal sites at onee (see Figure 8, 
Balancing Changes with Cause and Effect). 


108 



• When BITs are replaeed with outside vendors/analysts to measure results, 
their “outsider” replacement may not know the internal culture issues or the nuances of 
the enterprise, placing it at a disadvantage. 

• If consultants are hired, they charge steep fees for this type of service: six 
figures for larger companies or between $15,000 - 25,000 per month for smaller ones. 

Depending on the amount charged, outsourcing may be an acceptable solution for 
some companies that simply do not have the money and resources to invest in these 
skills. However the costs associated with this may risk achieving positive ROIMI 
returns, particularly for smaller companies. In addition, many organizations are 
uncomfortable with the service provider model because sensitive customer data is 
handled by a third party. Organizations are also at the mercy of the service provider for 
report customization and data retention policies. Generally, more money is saved by 
creating BITs in-house and the personal investments are greater and more reliable as 
well. Thus, corporations should consider seeking out its employees who best meet the 
BIT-related analytic skills needed (see Figure 14, Complimenting Skill sets and Tasks for 
BIT) for intranet portal analysis during the intermediate and developed terms because the 
right mix can create a competitive advantage. 

Teams selected to analyze Web metrics should try to keep choices tactical in 
support of strategic objectives, i.e., making changes in which the impact is generally 
known and can be measured tractably. For example, they must evaluate their success or 
failure by tracking metrics for each strategic objective, not solely metrics for each tactical 
initiative. By proceeding in this manner, they can overcome a variety of challenges (see 
below Figures 13, Top Challenges in Selecting and e-Business Project, and 14, Top 
Challenges Measuring Success of e-Business Investment) now confronting commercial 
and government sectors in trying to measure their intranet portals: 
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What are your top challenges in selecting 
which eBusiness projects to fund ? 


Integrating eBusiness into 
Overall Strategy 


Insufficient Resources 


Political Resistance 


Prioritizing According to 
Merit, not Hype 


Lack of Objective Data 



Percent of 50 eBusiness Executives Interviewed 
(Multiple responses accepted) 


Figure 16, Top Challenges in Selecting and e-Business Project 


What are your top challenges for measuring 
the success of eBusiness investments 


Attributing Incremental Value 
to Specific Initiatives 

Determining Customer 
Impact 

Lack of Objective Metrics 


Lack of Time to Evaluate 

Limited Technical Resources 



Percent of 45 eBusiness Executives Measuring Success 
(Multiple responses accepted) 


Figure 17, Top Challenges Measuring Success of e-Business Investment 


Cameron, 2000, p. 4. 
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D, DISCOUNTING 


BITs can also help separate soft benefits from the hard benefits any given projeet 
or enhaneement is expeeted to aehieve. In addition, they ean diseount soft benefits where 
appropriate. These diseounts help to hedge against rosy projeetions and ean be tailored to 
speeifie groups. For example, white-eollar knowledge workers may have their 
produetivity savings from software diseounted by up to 80 pereent, while faetory workers 
and salespeople may be diseounted by 20 pereent. The point being that any soft faetor 
ean be given a numerieal weight to aeeount for its understood and agreed upon impaet 
with regard to a speeifie line of work. Thus, while targeting high-end knowledge workers 
where the gains really matter most (see seetion V.D.2.d, High-End Knowledge Workers), 
BITs ean exereise diseretion in not over projeeting. However, if they “over-engineer” the 
proeess, people may walk away from the results. The predietors and weights used will no 
doubt beeome more aeeurate with praetiee gradually over time. This phenomenon — 
whieh is unique for every eompany — aeeounts for the laek of weights provided in the 
lEEM set of baseline metries. 

Along these lines, there are a variety of vendors supplying tools that measure 
metries in various ways. Unfortunately, few of these metries are indieators of business 
performanee and, taken out of eontext, these "standard" metries ean lead to poor 
eonelusions or no eonelusions at all. Henee, enterprises need to think hard about how 
they will apply Web analytics to achieve tangible business results. Creating their own 
Business Intelligenee Team to apply a set of baseline metries and relevant eonversion 
ratios that aeeount for eritieal business requirements, like those outlined within the 
framework of lEEM, is one method to start their measurement proeess in a standard, 
uniform manner through a single reporting serviee. 


" Cameron, 2000, p. 5. 
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E, SUMMARY 

In the final analysis, Web analytics are an immature discipline that requires the 
support and intelligence the human cognitive factors a BIT inherently brings. Within the 
lEEM framework, BITs can develop business cases that will help senior management 
better understand the value of a particular IT investment as it supports a variety of 
business requirements. Their task of tying metrics back to business justification is 
essential for determining success of intranets portals metrics and communicating the big 
picture. Naturally, the approach and empowerment (i.e., the extent to which they can 
recommend and enforce best practices across the enterprise based on their analysis) a BIT 
is allowed will depend in part on the style of the manager, the culture of the company, 
and support from top executives. Calculating and communicating their intranet analytic 
results will: 

• Enhance the standing of IT in eyes of business management. 

• Cut through culture issues. 

• Eead to better choices among potential IT portal initiatives. 

• Eead to yet better alignment of IT and business goals. 

• Provide feedback so IT can improve itself over time (hence, competitiveness). 

• Provide a forum and format that both facilitates and requires managers to 
detail how process changes will add business value. 

• Posters sponsorship and collaboration among departments and disciplines, 
preventing blind spots holding more people accountable. 

Along with the application of the metrics baseline and conversion rates to begin, adjust 
and continue measuring performance success, the complimentary skills and 
empowerment of BIT BDMs, portal managers and analysts ensure the azimuth indicator 
of lEEM remains pointed toward value: Internal awareness and responsiveness facilitates 
external awareness and responsiveness. 

In short, it's the process, and not technology, that makes for effective IT 
governance. The BIT works together to better orchestrate the impactful processes of 
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corporate portals. Before it gets involved in any tools, the BIT ensures processes are in 
place and understood by a wider audience at many levels, instilling discipline and 
acceptance. These proeesses involve inventorying IT resources, including skills, 
hardware, and software; analyzing their use by business goal, risk, budget, and expeeted 
return; and scenario planning. The eombination of process and technology in this manner 
by these people enables all knowledge workers to be more productive. 
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VII. CONCLUSION 


The newest innovations, whieh we label information teehnologies, 
have begun to alter the manner in which we do business and create value, 
often in ways not readily foreseeable even five years ago. 

Alan Greenspan, Chairman, US Federal Reserve 

A, SUMMARY 

The advent of intranet portals creates new opportunities for corporations to make 
capital investments in their own knowledge and streamlining of working processes at a 
rate of return far better than keeping the money in the bank. Proper metrics can both 
demonstrate the business value of intranets and portals as well as guide efforts toward 
enhancements that will have the greatest ROT Only firms that holistically consider 
organizational factors, information technology, and work processes will be able to 
identify such solutions to act upon. Because the intranet and its applications suites will 
have a measurable impact on traditional businesses, more companies must embrace a set 
of metrics that gives the IT initiative in this area credit not only for its customers, but also 
its overall contribution to the corporation at large. 

Organizations should predetermine the metrics they will collect, targeting 
customer loyalty, partner/affiliate assessment, content effectiveness, channel efficiency, 
etc. Web analytics tools help enterprises jump-start initiatives and provide inspiration, 
but the results are worthless without context. Along these lines, enterprises must develop 
a relevant plan of attack for measuring and monitoring their website data. By using the 
Intranet Efficiency and Effectiveness Model as part of a holistic approach toward a 
comprehensive appreciation of the interplay between business requirements and 
underlying intranet segments, constituents and relevant metrics, enterprises will gain 
valuable insight into how their websites are performing and how their users are 
interacting with their offerings. 

Invariably, a numerical model must be created and the most difficult part of this is 

identifying assumptions regarding costs and revenue generation. There is no hard and 
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fast rule for defining these assumptions. However, a eomprehensive model that stresses 
strategie, functional and technical fit, and seeks to identify opportunities for process 
improvements, is more likely to result in top management support because it is based on 
their direction for the organization, not merely the merits of a technology or another half- 
baked technique to manipulate numbers. 

The goal is to ensure that any IT investment decision can be shown to be 
consistent with the organization’s business objectives. It may take some experimentation, 
but eventually a set of metrics measurements can be created that, collectively and 
associated to the lEEM, describes different facets of how well a corporation is achieving 
its critical business requirements via its intranet portals initiatives large and small. These 
measurements should be codified and used consistently within a company; this will 
ensure that ROI results from different department portals are comparable to each other. 
Perhaps the most important benefit of applying the lEEM and metrics is that the business 
intelligence team will learn what they can do to better increase value through leading 
periodic re-examination of assumptions and results, applying experimentation, rewarding 
collaboration among IT, finance and business units, and promoting accountability. After 
all, a group can only improve when it has a chance to lean from both its mistakes and 
successes. It is the intent of the lEEM and its associated metrics analytics to be just such 
a starting place. 

B, CONTRIBUTION LIMITATIONS AND IMPLICATIONS 

This research provides the first theoretical model for the Family of Measures 
approach to measuring Web activity as well as a holistic framework and multi¬ 
disciplinary approach not previously derived in viewing and measuring intranet 
contributions in the context of a corporation’s overall strategic business requirements by 
applying a balanced baseline set of metrics and conversion ratios (which merge website 
traffic data and user behavior) linked to business processes as they relate to knowledge 
workers, portal managers and business decision makers. It also outlines how this should 
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be done and by whom in the form of periodie reviews and a business intelligenee team 
respectively. Lastly, it provides a means - as well as a justification — in which to 
calculate ROI for the metrics investment (time saved and better used by enhancements to 
processes) with a common unit of analysis: time, which can always be translated into 
money saved or earned and competitive advantage. 

However there are limitations that need to be taken into account in exercising and 
improving this model and approach. The lEEM metrics and BITs are intended primarily 
for large corporations who have multiple portals and a large number of knowledge 
workers. While the early promise of the Web was as a "great equalizer", the reality is 
that the Web exacerbates the advantages of scale. Although corporations with smaller 
intranets should not be dissuaded from using lEEM metrics, those with larger intranets 
will realize greater returns on the investment made into the metrics. At some point 
however, applying and realizing lEEM metrics benefits does not make financial sense if 
the intranet and initiative are small. 

Another limitation to this approach is that it takes time in order to measure 
changes sequentially and incrementally. In addition, if too many changes are made at 
once, their impact may become too diffuse or complex to estimate and trace credibly and 
accurately. At a minimum, it would take about 75 days before any comparisons can be 
made from two 30-day sets of results and preferably longer, one to two quarters with 
several 30-day result sets for example, before reliable and stable calculations/estimations 
are filtered and appropriate corrective action is taken (see Section IV.C: Periodic 
Review). The larger the organization, the longer the analysis will take. It would be wise 
therefore to begin small and incrementally include one or two portals at a time until all 
portals that need to be measured are. This gradual expansion, however, will require a 
longer period of time to canvas the corporation. In addition, the number and types of 
changes must be meticulously recorded and compared to ensure legitimate undesirable 
patterns are recognized and known remedies are effectively and immediately applied. 
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There may also be competing departmental portal interest with respect to the time 
and resources the BIT can apply toward taking corrective action. Such circumstances can 
become political and the BIT needs to be prepared to list and justify the enhancements 
they will undertake in both financial and effectiveness priorities (an extension of their 
own ROIMI and the lEEM metrics prioritization tables in Appendix D, Intranet Portal 
Metric Breakdown). As the number of portals and portal features increase the number of 
aggregate changes called for may also increase. Another surge in change requirements 
may be by necessity due to the advent of a new Web technology. Thus, BITs need to be 
empowered to make best practices that meet the dynamics of change and new business 
needs into common business rules and policies that can be applied and enforced in short 
order when and where necessary across the entire enterprise. Eack of this power is a 
serious limitation and will hinder progress toward achieving greater value. 

As portals mature they gradually borrow content from other portals and data- 
warehouses that they may have little control in changing. These sister content providers 
may be tasked to farm out their repositories of information (for example, news, research, 
archives or Human Resource related data). As a result, some portals owners may 
consider the lEEM metrics not to be a fair estimate of their value creation because they 
are getting a large percentage of their content from other internal sources to meet the 
demands of their knowledge workers. This situation is not necessarily a weakness in the 
lEEM because the metrics will determine which portals are the most trafficked and 
trusted and from exactly where and whom. This eventually leads to resource allocations 
and is why some portal managers fret over the notion that their significance may be 
diminished by a popular portal that does little else than provide news updates (a loose 
metaphor being the impact of CNN to syndicate channel ratings). Consequently, all 
assumptions and calculations along with concerns about comparisons between the 
participating portals need to be addressed in a reasonable and sensitive manner if the 
analysis results are to be perceived as valid by the portal players. Keeping all portal 
managers in the analytics loop is crucial therefore and should be scheduled routinely. 
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C. FUTURE WORK 

Successful Web analyties are more a matter of skills than a matter of technology. 
Nevertheless, Web logs need to be made as automated and quantitative as possible whieh 
presents a number of ehallenges to deeision makers when used for intranet 
measurements. Understanding eause-and-effeet is essential to the development of an 
aceurate appreeiation of user behavior, traffie volume statisties and a ROIMI. Many 
high-end Web analytic features (e.g., session analysis, multiple-site aggregation) offer 
online analytieal proeessing (OLAP) and data mining functionality via Java-scripted Web 
pages to eollect data. This relatively new teehnology provides significantly more 
information and seales better than proeessing of log files. One obstacle to seeing this 
through is that Java Server page teehnology doesn’t work (yet) with Active Server Page 
portlets - that means portal providers or integrators must adapt eontent to a format the 
portal understands sueh HTML, XML or Wireless markup language. Nevertheless, work 
to see how OLAP related teehnology ean be implemented into the lEEM metries analysis 
is highly desirable as it would serve as an enabler to deepen the analysis and shorten 
deeision and oorreetive aetion proeesses. 

As the find paradigm of the Web shifts from search to match (intelligent queries 
on a query so not to be under or over speeified), new teehniques will be required to 
ensure the proper metries are used to monitor which content nuggets are sought from 
where and to gauge the impaets on the user experienee. A promising teehnique that 
should be explored to assist in this endeavor is multivariate clustering-, a statistieal 
teehnique for dimensionality reduetion and eluster analysis applied to develop groups of 
similar online users based on eommonly held value eharaeteristies from among a baseline 
of value-driven variables/metrics. This technique explores different solutions eluster 
(and sub-eluster) baseline solutions. It should be further researehed to see if it eould 
serve as a viable trial for subsequent ehange reeommendations based on the previous 
metrics results. This type of researeh and analysis ean present more insightful and 
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diverse data in a shorter period of time with less disruption to the organization. As a 
result, the information gathered from this teehnique may lead to quieker, more aecurate 
adjustments to the value azimuth indieator of lEEM and eventually to eomplex 
algorithmie equations needed in the software engineer community which reflect the 
dynamics of the quality paradigm taking place in all segments and constituents that 
underscore value in corporate portal processes. 

Domain analysis for the Intranet is never completely finished. Product definitions 
evolve continuously. The development of a particular system that exploits previously 
accumulated domain knowledge can be the source for new insights about the domain that 
adds to or refines codified domain knowledge. As the multiple uses of Web analytics 
become apparent, new correlations will be blended to create unique value for each 
consumer of the information. There's no one metric that is right for all companies in all 
circumstances all the time. Inevitably, however, all measurements will continue to strive 
toward the quantification of benefits wherever possible. The role of software engineers 
in this endeavor will be to determine how best to ascertain value where it is not visible 
through domain analysis identification and implementation. lEEM provides a step in this 
direction as it aligns metrics to business priorities and intranet IT initiatives as well as the 
most relevant factors within all six underlying segments which have the most impact on 
deriving, measuring and increasing value. As portals become as ubiquitous in 
corporations as email is today, more inter-disciplinary research needs to be devoted in 
this area to help organizations of all types and sizes recognize, measure and capitalize the 
full potential of their greatest assets: customer data and intellectual knowledge. 
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APPENDIX A: CONSTITUENTS TO FINDING INTRANET INFORMATION 


PROBLEM; 

Knowledge workers eannot diseover all of the information they need to do their jobs 
better. 

QUESTION; 

What makes information more discoverable? 

REQUISITES; 

• Amount of information and meta-information (accessibility & manageability) 

• Understanding information seeking and use behavior 

• Navigation system design used to expose information (logically grouped) 

• Confidence in quality of information and meta-information 

• Relevancy to knowledge worker 

CONCEPT; 

“Discoverability”; - Eacilitate information discovery through presentation of 
relative associations which lead to better decisions, increased productivity and 
effectiveness that would not have been likely otherwise. Discoverability requires three 
events to happen; 1) associative connections between content items are created; 2) the 
information/ knowledge is presented to illustrate relative associations that can be acted 
upon; and 3) the knowledge worker has the cognitive ability/recognizes there is an action 
to be taken. 

CONSTITUENTS; 

Discoverability in an information system such as a corporate intranet is predicated 
and realizable if usable relative associations semantically exist. The term relative 
associations encompasses all types of relationships between items in an information 
system. These relationships are based on such things as characteristics of the items 
(properties), knowledge worker tasks and interactions between the items. Once 
established, these associations can be exposed and leveraged in an information system to 
enable a knowledge worker to move from concept to concept. 

Thus any portal desiring to be part of the information system should contain the 
following 13 constituents of discovering information that collectively sustain the 
requisites above and render relative associations useful. These constituents are grouped 
below into two categories or domains; “Back-end” - the processes that take place that a 
knowledge worker does not see; and, “Eront-End” - the processes in which a knowledge 
worker has a more direct interface. 
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CONSTITUENTS 

BACK-END 


INSTANCES OF 


EXPLANATION 


Content and Content 
Properties 

People, Organization, 
Tools, Application 
(scope is beyond 

documents) 

Covers: Search, Browse 
& Data Mgmt 

Content - All documents — help text, search results, 
forms, or application information — that is delivered to a 
knowledge worker via a website, or application. It is 
both subject and specific information. For example, 
“People” is a subject and a person by name is specific 
information. Content is organized and found depending 
on its properties and associations 

Content Properties - How content is described in order 
to affect the associations made in information retrieval 
and presentation. The characteristics of a content item 
make up its properties, such properties (a.k.a. attributes) 
include author, length, name, etc. If content is to be 
discoverable, it requires management of its properties 
and associations, as well as the tools and applications 
used to retrieve and present them. How well this is 
managed affects the overall effectiveness of any method 
of discovery, such as search and browse. 

Domain Integration 
Framework 

Domain Ownerships, 

Relationships, “Key 

Sites” 

Representation of the relationships/ structure between 
the key elements within a design. It shows how these 
elements interact and must “transfer” information, so 
that the architect can build the environment. Examples 
are information maps (document elements), server 
topography, scope of services, permissions (ownership). 

Information Life 

Cycle 

Expiring, Adding, 

Versioning, Retention 

The events often repeated again and again in 
maintaining the relevance and accessibility of content in 
an information system. These events include updating, 
versioning, archiving and, when necessary, deleting. 
(See: Retention in glossary). 

Search 

UI (Process and 

Presentation), Crawl 

An application employed by knowledge workers as a 
tool to find through direct surfacing or through surfacing 
an obvious navigational path. 

Taxonomy 

Vocabulary, Hierarchy, 
Meta-Data, Schemas, 
Heuristics from 

Longhorn? 

The result of identifying, creating and naming relative 
associations between items in an information system. In 
turn, the relative associations are grouped or classified 
into taxonomic structures based on three relationship 
types: hierarchical, equivalence and associative. 
Examples are controlled vocabularies; metadata scheme; 
category labels. (See: Relative Associations in 

glossary). 

User Data 

“My Data”, 

“My Profile”, 

Personalization, 
Segmentation, Metrics, 
Query Logs 

Facts and figures a knowledge worker maintains private 
access to for knowledge retention and expansion. This 
is also referred to as “personalization” (see below). 


Table 9, Back-End Constituents 
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CONSTITUENT INSTANCES OF EXPLANATION 


FRONT-END 

Accessibility 

Point of Access, 

Packaging, Formatting, 
Presentation 

Information is considered accessible when it is 
available, reachable, and understandable. (Assuming the 
information exists, ideally availability could occur from 
any point of access of and point of entry to the 
information system. Although how reachable 

information is depends on a number of factors, in this 
context the more information is compatible with a 
shared Information Architecture, the easier it is to 
retrieve. How this information is then packaged and 
presented in a fashion that can be understood by the 
knowledge worker is the last stage of accessibility.) 

Communication of 

Authoritativeness and 
Importance 

Site Maps, Credibility 
(Best Bet), Authoritative 
Flag, Relevancy 

Communicating to the knowledge worker the credibility 
of an information system within that system to ensure 
confidence and trust. See: Authoritative Flags. 

Communication of 

Understanding 

Links, Definitions, 

Mouse-over, Tool Tips, 
Contacts, Smart Tags, 
Context 

Communicating to the knowledge worker meaning and 
significance to ensure acceptance and engagement, 
which further increases the audience’s propensity to act 
on information in a common way, i.e., the Glossary of 
IP Terms and Definitions, Mouse-over, and best 
practices. 

Information Grouping 
& Segmenting 

Content Nuggets, 

Categories, Applications 

Information grouping is the logical collection of similar 
and relevant information, such as Content Nuggets and 
Categories respectively. Another aspect of this is 
information segmenting in which only the relevant parts 
of a document, i.e. a paragraph or two are extracted and 
placed into fragmented groupings instead of the entire 
document. For example, first 200 words of a document 
returned as part of a search result. 

Navigation 

- Local 

- Global 

Presentation 

Consistent Labeling 
Sources of information, 
applications 

Method of moving through the domain framework by 
way visual presentation and consistent choices. 
Navigation has 2 basic types: local (also known as 
vertical) and global (also known as horizontal). 

Local navigation presents choices leading to subtopics 
or sub-areas of a site, usually defined by one of its main 
menu subjects (often referred to as a “drill down”). 

Global navigation presents choices leading to other main 
areas of a site, i.e. Home; Search; About. Global 
navigation is consistent throughout an information 
system which allows knowledge workers to go across 
portals with a sense of familiarity. 

Personalization and 
Customization 

Security, Content 

Filtering, 

Authentication, 

Personalization - A method of contextualizing 
information for a knowledge worker based on what is 
known about the knowledge worker. A site can use 
personalization to alter navigation and content presented 
according to the perceived needs of the knowledge 
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worker. Examples include content filtering based on 
role; authentication based on name or status as a 
manager. 

Customization - Ability for knowledge workers to self- 
configure the contents/ constituents of their domain 
site. In essence, individuals are allowed to subscribe to 
constituents, or sub-constituents of a portal, including 
navigation, content, search, etc., in order to provide and 
store information (i.e., like the notion of “my saved 
searches”) most relevant to their interests, position or 
role. 

User Assistance 

Help in context, 

Feedback, Dialog with 
the knowledge worker. 
Training 

Help made available to the knowledge worker while 
using an information system. User assistance provides 
guidance on how to use the system or additional help in 
finding information sought. Examples are help in 
context, feedback. Dialog boxes, and training, real time 
assistance from a human intermediary via a live chat. 


Table 10. Front-End Constituents 
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APPENDIX B: ONLINE E-SURVEY EXAMPLE 


Useful general e-survey questions about intranets and employee portals inelude: 

One minute of your valuable input will help the company provide you with better intranet 
service. Please take a moment to complete this automatic survey as part of our on-going 
effort to optimize out intranet according to your needs: 

1- means seldom, very little, poor, no 
5- means very often, very much, exception, yes 


What is the most useful feature of the portal for you? (To be chosen from 
provided list of portal features.) 

A. ) 

B. ) 

C. ) 

Which of the following processes do you use the portal to accomplish? 
(To be chosen from provided list of portal processes.) 

A. ) 

B. ) 

C. ) 


1-2-3-4-5 Which new feature would increase your portal usage the most? (To be 
chosen from menu of potential new portlets.) 

A. ) 

B. ) 

C. ) 


How can the portal be improved to enable employees to be more 

productive? 


1-2-3-4-5 


1-2-3-4-5 


1-2-3-4-5 


1-2-3-4-5 


How critical is the portal to performing your job function? 
How up-to-date is the information in the portal? 

How would you rate the performance of the portal? 

Has the portal met or exceeded your expectations? 


1-2-3-4-5 How relevant is the information in the portal to your job? 


1-2-3-4-5 


1-2-3-4-5 


1-2-3-4-5 


Is the portal part of your daily routine? 

How would you rate ease of use of the portal? 

How many times per day (or what percentage) do you use the portal? 


SUBMIT 

THANK YOU 

(Note: To be useful, the survey must capture timing and demographic information 
about the respondent (see Figure 2, Intranet Domains and Segments). When this 
information cannot be determined automatically from a respondent’s profile, the 
respondent should be queried directly. ) 
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APPENDIX C: METRICS TO MEASURE INTRANET PERFORMANCE 


The following table lists numerous specific 

Revenue Eorecast 

metrics that can be applied to measure a 

Qualified Revenue 

portal site’s activity - not all of them are 

non-Qualified Revenue 

needed for an intranet. When a subset of 

Visits 

these metrics are further logically 

Qualified Visits 

represented and grouped in agreement with 

non-Qualified Visits 

the segments and constituents that the 

Product Qualifications 

lEEM outlines in support of assessing 

Visitor Conversion Ratio 

critical business requirements, rational 

Results by Visitor Qualification 

subsequent and impactful actions can be 

Results by Product Category 

taken based off these metric results which 

Results by Product 

lead to coherent and sensible value gains 

Cost 

(see Appendix D for the baseline approach 

Summary 

to do this). 

Per Visit 

-represents a metric category 

Per Qualified Visit 

Grey -represents a useful lEEM metric 

Per non-Qualified Visit 


Per Product Qualification 



ROI 

General Statistics 


Summary 

Page Views Over Time 


Per Day 

Top Pages by Visits 


Per Visit 

Advertising Click Through Rate 


Per Qualified Visit 

Hits Over Time 


Per non-Qualified Visit 

Visitors Over Time 


Per Product Qualification 

Top Visitors 


Projections 

Top Referring Sites by Visits 


Revenue Eorecast 

Top Browsers by Visits 


ROI 

Commerce Analysis 


Visit 

Summary 


Qualified Visit 

Executive Summary 


non-Qualified Visits 

Results Summary 


Product Qualifications 

Abandonment Summary 


Referrers 

Results by Marketing Campaign 


Revenue Eorecast 

Shoppers 


Qualified Revenue Eorecast 

Buyers 


non-Qualified Revenue Eorecast 

Attempted Buyers 


Visits 

Engagement Rate 


Qualified Visits 

Conversion Rate 


non-Qualified Visits 

Abandonment Rate 


Product Qualifications 

Summary by Marketing Campaigns 


Results by Visitor Qualification 

Marketing Campaigns 


Results by Product Category 

Results 


Results by Product 

Summary 


Product Categories 
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Table 9 - Specific Metrics 









Revenue Forecast 

Product Qualifications 

Results by Product 

Results by Visitor Qualification 

Results by Marketing Campaign 

Results by Referrer 

Products 

Revenue Forecast 
Product Qualifications 
Results by Visitor Qualification 
Results by Marketing Campaign 
Results by Referrer 
Products Summary 


Top Pages 

Top Pages by Visits 

Top Pages by Visits Over Time 

Top Pages by Views 

Top Pages by Views Over Time 

Top Documents 

Top Documents By Visits 

Top Documents By Visits Over Time 

Top Documents By Views 

Top Documents By Views Over Time 

Dynamic Pages & Forms 

Dynamic Pages & Forms by Visits 

Dynamic Pages & Forms by Visits Over 

Time 

Dynamic Pages & Forms by Hits 
Dynamic Pages & Forms by Hits Over 
Time 

Page Views Over Time 

Top Content Groups 

Top Content Groups by Visits 

Top Content Groups by Visits Over Time 

Top Content Groups by Hits 

Top Content Groups by Hits Over Time 

Top Entry Pages 

Top Entry Pages by Visits 

Top Entry Pages by Visits Over Time 

Eeast Requested Entry Pages 

Eeast Requested Entry Pages by Visits 

Eeast Requested Entry Pages by Visits 

Over Time 

Top Exit Pages 

Top Exit Pages by Visits 


Top Exit Pages by Visits Over Time 

Single Access Pages 

Single Access Pages by Visits 

Single Access Pages by Visits Over Time 


Top Paths Through Site by Visits 
Top Destination Paths Through Site 


Hits Over Time 

Top Directories 

Top Directories by Visits 

Top Directories by Visits Over Time 

Top Directories by Hits 

Top Directories by Hits Over Time 

Top Directories by Kbytes Transferred 

Most Downloaded Piles 

Most Downloaded Piles 

Most Downloaded Piles Over Time 

Most Downloaded Piles by Visits 

Most Downloaded Piles by Visits Over 

Time 

Most Accessed Pile Types 

Most Accessed Pile Types 

Most Accessed Pile Types by Kbytes 

Transferred 

Most Uploaded Piles 

Most Uploaded Piles 

Most Uploaded Piles by Visits 

Top Entry Piles 

Top Entry Piles by Visits 

Top Entry Files by Visits Over Time 

Eeast Requested Entry Piles 

Least Requested Entry Piles by Visits 

Least Requested Entry Piles by Visits 

Over Time 


URL ID Parameter Analysis by Visits 
URL ID Parameter Analysis by Hits 
URL 2D Parameter Analysis 


Advertising Click Through Rate 
Ad Visits 
Ad Visits 

Ad Visits Over Time 
Ad Views 
Ad Views 
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Ad Views Over Time 
Ad Visits With Clicks 
Ad Visits With Clicks 
Ad Visits With Clicks Over Time 
Ad Clicks 
Ad Clicks 

Ad Clicks Over Time 


Top Visitors 

Top Visitors 

Top Visitors Over Time 

Top Visitors by Hits 

Top Visitors by Hits Over Time 

New vs. Returning Visitors 

New vs. Returning Visitors 

New vs. Returning Visitors Over Time 

Top Authenticated Visitors 

Top Authenticated Visitors 

Top Authenticated Visitors Over Time 

Top Authenticated Visitors by Hits 

Top Authenticated Visitors by Hits Over 

Time 

Visitors Over Time 
Visitors Over Time 
Visits Over Time 
Visitors by Number of Visits 


Top Geographic Regions 

Most Active Countries 

North American States and Provinces 

Most Active Cities 

Most Active Organizations 

Most Active Organizations by Visits 

Most Active Organizations by Hits 

Top-Level Domains Types 

Top-Level Domains Types by Visits 



Summary of Activity for Report Period 
Visits by Number of Pages Viewed 
Bandwitdh:Kbytes Transferred Over Time 
Average Time to Serve Documents 
Average Time to Serve Dynamic Pages 
Day of Week 
Visits by Day of the Week 
Hits by Day of the Week 


Hour of Day 

Visits by Hour of the Day 
Hits by Hour of the Day 
Length of Visit 
Visits by Length of Visit 
Page Views by Length of Visit 


Server Cluster Load Balance by Hits 
Server Cluster Load Balance by Kbytes 
Transferred 


Technical Statistics and Analysis 
Dynamic Pages & Forms Errors 
Client Errors 
Server Errors 


Top Referring Sites by Visits 
Top Referring UREs by Visits 
Top Search Engines 
Top Search Phrases 
Top Search Keywords _ 


Top Browsers 

Top Browsers by Visits 

Top Browsers by Visits Over Time 

Top Browsers by Hits 

Top Browsers by Hits Over Time 

Microsoft Explorer Browsers 

Microsoft Explorer Browsers by Visits 

Microsoft Explorer Browsers by Visits 

Over Time 

Microsoft Explorer Browsers by Hits 

Microsoft Explorer Browsers by Hits 

Over Time 

Netscape Browsers 

Netscape Browsers by Visits 

Netscape Browsers by Visits Over Time 

Netscape Browsers by Hits 

Netscape Browsers by Hits Over Time 

Top Platforms 

Top Platforms by Visits 

Top Platforms by Visits Over Time 

Top Platforms by Hits 

Top Platforms by Hits Over Time _ 


Table 11, Specific Metrics 
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APPENDIX D: INTRANET PORTAL METRIC BREAKDOWN 


The ensuing tables of baseline metrics and conversion ratios is color coded in line with 
the illustration of the model below to highlight from which domain and metric type metrics 
originate. The model is an approach to determining effectiveness taking into account the 
pertinent characteristics of intranet domains, segments and constituents with the resulting 
supposition delivered by a variety of metrics specified in priority and divided into audiences and 
responsibilities according to business requirements. 


Front-End 


Easier to Measure 


Back-End 


Sat. Surveys 

.UE 

No. Clicks to Find 
Tie to Locate 


Activity by 

llcanA Frequent 
Time, & usage 

Duration Traffic Volume 


Unique Users 



People, Process, Technology 


Corresponding 
Color Codes on 
Metric Baseline 
Tables 



Business Req. or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


Figure 18, lEEM Color Code Supporting Metrics Baseline Tables 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 

Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


WHY: 


HOW: 




<- 

—business process- 

-X - best practices — 


<- 

— lEEM Related 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Soecific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Scementts): 

lEEM Constituent(s) 

Efficiency (E) 
or 

EfTectiveness(F) 

Suceess: 

- Visitors (by 
Search Engine) 

- Downloads 
(by Search 
Engine) 

Which configuration 
for our search engines 
should I pay attention 
to? 

How a chosen search 
engine is configured takes 
time and money to 
develop. Configurations 
must be justifiable in terms 
of relevant, helpful finds 
and downloads or 
takeaways, which lead to 
better decisions and 
opportunity, leading 
ultimately to increased 
revenue and understanding 
the ROI. Thus 
determining which 
configuration (including 

Best Bets and Best 

Practices) provides the 
best relevance is critical to 
enabling users to find the 
information they need to 
do their jobs. 

Visitor bv Search En2. Confi2uration 

- Top Referring Sites by Visits 

- Top Documents By Visits 

- Top Content Groups by Visits 

- Top Entry Pages by Visits 

- Top Directories by Visits 

Downloads bv Search Engine 

- Top Documents By Views 

- Top Content Groups by Hits 

- Top Directories by Kbytes Transfer 

- Top Search Engines 

ROI bv Search Engine Configuration 

- Best Bet: Most relevant hits 

- Successful Approach 

- User Survey: User Feedback 

User - Find more frequently infonnation they need 

Portal - Product Manager provides suggestions (in the 
form of collective Best Bets and Best Practices) to 
search configuration algorithms 

Enterprise - When users and portals benefit from more 
relevant search hits, so does the enterprise in aggregate 

Info. Architecture 

- Search 

F 


WHAT: RELEVANCE (Priority No. 1) 

WHERE: Enterprise 

WHO: Portal Owner Related 


Table 12, Relevance Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: TRAFFIC VOLUME (Priority No. 2) 

WHERE: Enterprise 

WHO: Portal Owner Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS 


><-best practices-> 


<-lEEM Related-> 


Business Issue: 

- Metric Area 


Business Question of Business Significance of 
Web Site Activity Web Site Report Solution 


Specific Snb-Metric Area 

- Specific Metric 


Wbo Benefits and Wby 

(User, Portal, Enterprise) 


lEEM Segmentts): 

lEEM 

Constituent(s) 


Efficiency (E) 
or 

Effectiveness(F) 


Usefulness 
(Leads Commit. 

and Completed ): 

- VaM 

- 

- Loycilhi 


How many visitors is my 
web site producing? Are 
we converting visitors to 
committed leads, i.e., an 
exchange of information, 
and completed processes, 
i.e., downloads? 


Identify and track leads by 
web browsing behavior to 
understand total quantity 
and quality of leads. 
Quantify lead to download 
conversion rates (or length 
of site visit duration) to 
understand lead generation 
effectiveness in order to 
better understand what 
adjustments need to be 
made (i.e., site re-design, 
elevation of highly sought 
after pages or click to 
success and pages to drop). 


Visitors to Leads 

- Top Pages by Views Over Time 

- Top Documents by Views Over Time 

- Page Views by Length of Visit 

- Visits by Length of Visit 

- Top Exit Pages by Visits 

Total Leads 

- Most Accessed File Types 

- Most Uploaded Files 

- Top Directories by Hits Over Time 

- Top Content Groups by Hits Over 
Time 

Lead to Download Conversion 

- Most Downloaded Files by Visits 

- Most Downloaded Files by Visits Over 
Time 


User - Exchange information only when necessary 
and download or view (time duration) what is 
needed. 


Info Architecture 
- Domain Integrated 
Framework (DIF) 


Portal - Managers learn where visitors fail to make 
leads or where leads fail to complete the process and 
can make changes accordingly (analysis can be 
provided by Product Manager) 


Design 

- Usability Study 


Enterprise - When users and portals benefit from 
visits becoming leads and leads being completed in 
the form of downloads or views, the enterprise 
benefits as well by virtue of users making better 
decisions from finding what they need. 


F 


Usability Study 

- Usability 


Table 13. Traffic Volume Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: 

UNIQUE USERS (Prioritv No. 3) 

WHERE: 

Enterprise 

WHO: 

Portal Owner and Portal Manager Related 

WHY: 

HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Metric Area 

- Specific Metric 

Who Beneflts and Why 
(User, Portal, Enterprise) 

lEEM ScemcntlsI: 

lEEM Constituent(s) 

Efficiency (E) 
or 

ElTectivenessCF) 

Retention: 

How effectively am I 

Determine how quickly 

Return Visitor Rate 

User - Gains familiarity to a site which reduces overall 

Info Architecture.: 


- 

Eova/d 

ValiiJT 

building loyalty with 
my visitors? 

you are building your user 
base to gage site audience 
enlargement and shrinkage 
over time and vis a vis 
other sites. 

- Top Visitors (authenticated) 

- Visitors by Number of Visits 

- Visitors Over Time 

- Top Visitors by Hits (Leads) 

- Top Visitors by Hits Over Time 

- Top Visitors by Leads Completed 

- Visits by Length of Visits 

* All of the above for Returning Visitors 

- Hew vs. Returning Visitom 

frustration. 

Portal - Managers learn where return visitors come 
from and how many there are, which a partial indication 
that users find the site helpful. 

Enterprise - When users continue to return to the same 
site for information, it indicates that they find it helpful 
and s well as where the sources of richness lie. 

-User Data 

F 

Ontimization: 

What do my visitors 

Analyze the most popular 

Return Visitor Tarset Paecs 

User - Popular downloaded information remains as well 

Info. Knowledge 



LoyaiM 

ValuJT 

^timizatio4 

come back for? 

content for my return 
visitors in order to load 
and associate related 
information to meet 
demand. 

(correlate return visits with content): 

- Top Returning Visitors by Hits 

- Top Returning Visitors by Hits Over 
Time 

- Top Returning Visitors by Leads 
Completed 

- Returning Visitor Visits by Length of 
Visits 

- Returning Visitor Page Views Over 

Time 

- Top Document and Content Group for 
Returning Visitors Over Time 

Survey 

- Internal Returning Visitor Session 
Activity 

as other information like it eventually being posted or 
better associated. 

Portal - Managers learn which pages are desired and 
can load related information to meet demand. 

Enterprise - As portals more accurately monitor what 
information is sought and add additional information 
that is related, the enteiprise is doing a better job 
meeting the expectations and business needs of its users. 

- Survey 

Info Architecture 

- Info Grouping 

- User Data 

F 


Table 14, Unique User Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: 

UNIQUE USERS tPriorltv No. 3) 

WHERE: 

Enterprise 

WHO: 

Portal Owner and Portal Manager Related 

WHY: 

HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS 


><—best practices-> 


<-lEEM Related-> 


Business Issue: 

- Metric Area 


Business Question of Business Significance of 
Web Site Activity Web Site Report Solution 


Specific Snb-Metric Area 

- Specific Metric 


Who Benefits and Why 

(User, Portal, Enterprise) 


lEEM Segmentts): Efflciency (E) 

lEEM Constituent(s) Eltectivenes.CF) 


Targeted ; 

- Itc^cA 

- 

- ValuJ[ 

- Disintermedi¬ 
ation 

- Access 


Where do my visitors 
come from 
(specifically, which 
regions, organizations, 
groups, roles)? 


Identify where users 
originate and how your 
web site is searched and 
browsed to understand 
what content areas are 
most effective to improve 
overall productivity for a 
particular region 
organization, group and/or 
role). 


Users by Region 

- Most Active by Visits 

- Most Active by Hits 

- Most Returning Visitors 

Users by Group 

- Most Active by Visits 

- Most Active by Hits 

- Most Returning Visitors 

Users by Role 

- Most Active by Visits 

- Most Active by Hits 

- Most Returning Visitors 


User - Potentially better personalized service 

Portal - Managers learn which pages are desired and 
most effective by region, organization, group and role 
and can make enhancements accordingly. 

Enterprise - As portals more accurately monitor what 
information is sought and add additional information 
that is related, the enterprise is doing a better job 
meeting the expectations and business needs of its users, 
both enterprise and world wide. 


Info. Knowledge 

Info Architecture: 

-DIF 

-Taxonomy 
-Content Properties 
-Accessibility 

Ed. Programming 

- Usability 

- Ease of Navigation 


F 


Usability Study 

- Internal Visitor Session Activity 

- Usability 


Table 14, Unique User Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: CROSS PORTAT. REFERRAT. (Priority No. 4) 

WHERE: Enterprise 

WHO: Portal Owner Related 

WHY: HOW: 


Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 


Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecifie Snb-Metrie Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Seement(s): 

lEEM Constituent(s) 

Efficiency (E) 
or 

EfTectiveness(F) 

Aehievement: 

- 

- Disintermedi¬ 
ation 

Is my web site 
becoming a more 
important channel for 
information? 

By measuring customer 
satisfaction and the visitors 
(as well as subsequent 
leads) from others web 
sites to yours, you will be 
able to understand your 
web site’s importance and 
relevance. 

Visitor bv Source (Referral from other 

Portal) 

- Top Referring Sites by Visit 

- Top Referring URLs by Visit 

Leads bv Source (correlate visitor bv 

source refen'al to lead) 

- Top Referring Sites by Visits to Leads 

- Top Referring URLs by Visits to Leads 

Leads Completed bv Source (correlate 

visitor bv source to lead comnleted) 

- Top Referring Sites by Visits to Leads 
Completed 

- Top Referring URLs by Visits to Leads 
Completed 

Uniaue and New Users 

- New Vi'. Returning Visitors 

- New Vi. Returning Visitors Over Time 

Survey 

- User Sat. 

User - Site provides or directs you to a site with the 
needed information and user has opportunity to express 
satisfaction. 

Portal - Managers learn which pages are desired and 
can load related infonnation to meet demand. Mangers 
learn visitor, lead and lead completed ratios by source to 
determine most valuable referral sites. 

Enterprise - Ultimately, the more referrals the more 
information is to be discovered. As portals are 
monitored for referral by source, the value of a portal 
takes on an added dimension to overall significance or 
enhancer to enterprise information discovery (intranet 
portals will in effect be both a beneficiary and 
benefactor of referrals). 

Info Architecture: 

-Navigation 

-Search 

Info Knowledee 

User ExDerience 

- Sat. Survey 

F 


Table 15, Cross Portal Referral Metrics to Business Requirements 

(Continued on Next Page) 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: CROSS PORTAT. REFERRAL (Priority No. 4) 

WHERE: Portal 

WHO: Portal Manager Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 

Practice 

rracnce 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snccific Snb-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Sc2mcnt(sf: 

lEEM Constituent(s) 

Efficiency (E) 
or 

EITectivenesstF) 

Ootimization* 

- Survey 

How can I maximize 
the effectiveness of my 
online services 
offerings? 

Measure the visitation to 
online service resources 
and inform services people 
of key areas of concern or 
need as a mechanism to 
improve business 
performance. 

Web Pa2e Most Visited: 

- Top Pages by Views Over Time 

- Top Documents by Views Over Time 

- Page Views by Length of Visit 

- Visits by Length of Visit 

- Top Exit Pages by Visits 

- Most Accessed File Types 

- Most Uploaded Files 

- Top Directories by Hits Over Time 

- Top Content Grps by Hits Over Time 

- Most Downloaded Files by Visits 

- Most Downloaded Files by Visits 

Over Time 

Load Balance and Cachin2: 

- Server Cluster Load Balance by Hits 

- Server Cluster Load Balance by 

Kbytes Transferred 

Survey 

- User Feedback: 

User - Sites are proactive in monitoring what your 
interests are in. 

Portal - Managers learn which pages are desired and 
can load related information to meet demand. 

Enterprise - More responsive and pro-active in 
providing overall Intranet support, leading to increased 
user sat. 

Info Knowlecl 2 e; 

- Survey 

Info Architecture: 

- User Data 

F 

Usefulness: 

- User Sat. 

^reativit.\i 

Pfc/.v/ort 

Which user segments 
utilize web service 
pages? 

Anticipate possible service 
issues and recommend 
solutions based on 
segmentation of user type 
and web activity. 

User Se2nient Usa2e Rate: 

Users bv Grono 

- Most Active by Visits 

- Most Active by Hits 

- Most Active by Duration of Visit 

- Most Active by Download 

- Most Returning Visitors 

User - Sites are proactive in monitoring what your 
interests are in (and provide some personalization). 

Portal - Managers learn user behavior and needs and 
attempt to anticipate what information is desired to the 
lowest denominator possible. 

Enterprise - More responsive and pro-active in 
providing overall Intranet support, leading to increased 
user sat. 

Info Architecture: 

- User Data 

F 


TABLE 15, Cross Portal Referral Metrics To Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: CROSS PORTAT. REFERRAT. (Priority No. 4) 

WHERE: Portal 

WHO: Portal Manager Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Scemcntls): 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectiveness(F) 

Preservation: 

- User Sat. 

- ^aentiot 

How do I increase user 
loyalty by proactively 
contacting my users? 

Deliver personalized 
services offerings (via UI 
or email) directed toward 
identified user needs 

Reneat Visits 

- Top Visitors (authenticated) 

- Visitors by Number of Visits 

- Visitors Over Time 

- Top Visitors by Hits (Leads) 

- Top Visitors by Hits Over Time 

- Top Visitors by Leads Completed 

- Visits by Length of Visits 

* All of the above for Returning Visitors 

Personalization 

- Automated and User Determined 
Personalization 

Survey 

- Emails to High Frequency Visitors to 
Solicit Needs 

User - Sites are proactive in monitoring what your 
interests are in (and provide some personalization). 

Portal - Managers learn user behavior and needs and 
attempt to anticipate what information is desired to the 
lowest denominator possible. 

Enterprise - More responsive and pro-active in 
providing overall Intranet support, leading to increased 
user sat. 

Info Architecture: 

- Personalization 

- User Data 

User Exnerience: 

- Personalization 

F 

-Dintermedia- 

tion 



TABLE 15, Cross Portal Referral Metrics To Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: CROSS PORTAL REFERRAL (Priority No. 4) 

WHERE: Enterprise 

WHO: Portal Manager and Editor Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Scemcntls): 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectiveness(F) 

Retention: 

- User Sat. 

- Usability 

- Ease of 
Navigation 

Why are visitors 
leaving our site, even 
though they have 
already selected 
several items or started 
a lead session? 

Monitor dropped pages 
and then study them to 
understand why they were 
dropped and make 
modifications (i.e., design 
change) to improve 
discovery of information 

Abandons: 

- Dropped page rate over time 

User Sat. Survey: 

- Abandonment causes 

Usability Study: 

- Abandonment causes 

User - Pages where users have problems will be 
identified and users may have chance to comment on 
their modification. 

Portal - Managers learn which pages are desired and 
can load related infonnation to meet demand. Mangers 
learn visitor, lead and lead completed ratios by source to 
determine most valuable referral sites. 

Enterprise - Ultimately, the more referrals the more 
information is to be discovered. As portals are 
monitored for referral by source, the value of a portal 
takes on an added dimension to overall significance or 
enhancer to enterprise information discovery (intranet 
portals will in effect be both a beneficiary and 
benefactor of referrals). 

Ed. Programming. 

User Exnerience: 

- Survey/Feedback 

F 


TABLE 15, Cross Portal Referral Metrics To Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: EASE OF NAVIGATION (Priority No. 5) 

WHERE: User 

WHO: Portal Manager and Editor Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 



Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Scemcntts): 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectiveness(F) 

Site Design 

Is our site navigation 
effective? 

Measure how difficult it is 
for your visitors to take the 
desired action. 

Clicks and Duration Measures: 

Usability Study: 

User - Pages. 

Portal - Managers learn. 

Enterprise - 

Ed. Proarammine. 

User Exnerience: 

- Navigation 

F 

Site Design 

How do I know if my 
site is optimized for 
my typical web site 
visitor? 

Track which browsers and 
platforms your visitors use 
so you can design 
according to their needs 

Web Graphics: Referrers, Browsers 
used, Search Engines 

User - Pages. 

Portal - Managers learn. 

Enterprise - 

Ed. Proarammina. 

User Experience, 
Info. Architecture 

- Personalization 

F 

Site Design 

How can I make 
transactions easier for 
my visitors? 

Analyze the seeking 
processes to determine 
speed the process to 
download 

Click to Success Rate 

User - Pages. 

Portal - Managers learn. 

Enterprise - 


E(F) 



Ed. Proarammina. 

User Exnerience: 

- Personalization 

Site Design 

How do I evaluate the 
success of design 
tradeoffs and choices? 

Continuously measure the 
effect of each design 
change on your visitation 

Caebin2 Statistics 

User - Pages. 

Portal - Managers learn. 

Enterprise - 

User Exnerience. 

Ed. Proarammina: 

- Navigation 

Info. Knowledge 

F 


Table 16, Ease of Navigation Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: TOP DOWNLOADS / PAGES (Priority No. 6) 

WHERE: Portal 

WHO: Portal Manager Related 

WHY: HOW: 




Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Scemcnttsl: 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectiveness(F) 

Site Design 

Are there popular 
pages that aren’t easy 
to access? 

Improve access to popular 
pages and satisfy your 
visitors 

Deen Link Correlation tcontent sroun 

bv no referreri 

User - Pages. 

Portal - Managers learn. 

Enterprise - 

Info Architecture: 

- Info Grouping 

- Search 

- Content Properties 

F 

Site Design 

What keywords should 

I use to maximize 
search engine traffic to 
my site? 

Maximize traffic to your 
site by understanding what 
terms your visitors are 
using to locate your site 

(Search engine optimizer) 

User -Pages. 

Portal - Managers learn. 

Enterprise - 

Info Knowledge: 

- Content Properties 

F(E) 


Table 17, Top Downloads/Pages Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: USER SATISFACTION SURVEYS (Priority No. 7) 

WHERE: Front End 

WHO: Portal Owner, Manager, Editor and User Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

Snecific Snb-Mctric Area 

- Specific Metric 

Who Benefits and Why 
(User, Portal, Enterprise) 

lEEM Seamentfsl: 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectlveness(F) 

User Experience 

Do users prefer and 
use my site? Why and 
Why not? 

Improve satisfaction of 
your content delivery to 
help users do their jobs and 
make decisions better. 

User Sat Survey with a variety of 
questions related to the business 
discussion 

Customer 

Portal 

Enterprise 


All Seements 

- All lEEM 
Constituents 

F(E) 



Site Design 

How can the site (UI) 
be modified to best 
meet the demands of 
users? 

Improve satisfaction of 
your content delivery 

User Sat Survey with a variety of 
questions related to the business 
discussion 

Customer 

Portal 

Enterprise 


All Sesments 

- All lEEM 
Constituents 

F(E) 



Table 18, User Satisfaction Survey Metrics to Business Requirements 
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Intranet Portal Metric Supposition and Breakdown on Measuring Intranet Websites to Indicate and Improve Effectiveness Gains 


WHAT: Time to Locate (Priority No. *) 

There are many variables that eould skew the results for time to reaeh a deeision to be eonsistent (i.e., individual skill sets and experienee ean vary 
greatly). However, if improvements are made in the other metrie areas previously listed, they will eollectively help to minimize the time to loeate 
desired information and the resulting deeision reached. Nevertheless, from time to time users may find what they are looking for sooner, but will 
also continue to look for long periods of time (perhaps as much as the approximate 50% of their time as they do now) because they are finding more 
of what they are seeking. Regardless, the time factor is reduced with respect to finding what is sought or considered desirable: If people still spend 
50% of their time looking for information, they should have more pertinent information than before in the same amount of time - which should lead 
to better decision making and ultimately more effectiveness. “Time to Locate”: is an efficiency metric that is affected by increases in effectiveness 
elsewhere. The efficiency metric of Time to Locate in turn affects effectiveness across the board because users will either have or can do more in less 
time. Thus, this metric area is an example of how effectiveness affects efficiency and then how this efficiency increase in turn improves 
effectiveness. 
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WHAT: Miscellaneous 

WHERE: Portal 

WHO: Portal Manager and Editor Related 

WHY: HOW: 



Business Req, or 
Derived Metric 

Back End or 
Hard Metric 

Front End or 
Soft Metric 

Best Bet or 
Practice 


<-BUSINESS PROCESS-><-BEST PRACTICES-> <-lEEM Related-> 


Business Issue: 

- Metric Area 

Business Question of 
Web Site Activity 

Business Significance of 
Web Site Report Solution 

SDCcific Sub-Metric Area 

- Specific Metric 

Who Benefits and Why 

(User, Portal, Enterprise) 

lEEM Seementts): 

lEEM Constituent(s) 

Efficiency (E) 
or 

Effectiveness(F) 

User Experience 

How often do people 
abandon my 
registration and survey 
forms? 

Monitor incomplete survey 
and registration forms and 
analyze why they were 
abandoned 

Registration abandonment rate 

Portal 

User 

Info. Knowledee 

E(E) 

User Experience 

How often do people 
bookmark my site? 

Understand how important 
your content is to certain 
visitors 

Bookmark volume trend 

Portal 

User 

I 

nfo. Knowledge 

F 

Performance 

How do I gauge the 
performance of my 
caching or load 
balance? 

Analyze and understand 
what perfomiance benefit 
caching has for my visitors 
and make appropriate 
changes 

Caching statistics 

Load Balance statistics 

Enterprise 

Portal 

User 

Info. Knowledee 

E 

Web Site 
Performance 

How do I know what 
errors are occurring on 
the web site? 

Control error rates by 
tracking client, server, and 
dynamic page/form errors 
and fixing them 

Visitor Error Rate 

Portal 

Info. Knowledee 

E 

Table 19, 

Miscellaneous Metrics to Business Requirements 


Portal Manager Questions Related to lEEM Buy-in (Miscellaneous) 


Business Issue 

User Pain 

Interoperability 

Will this solution work with my existing environment? (Databases, applications and hardware platfomis) 

Data Integration 

How do you integrate multiple data sets, on multiple platforms for multiple people? 

Productivity 

How can I automate the process of distributing the analysis and reports on website activity? (and present them in a format that is understandable and meaningful) 

Scalability 

Can the system scale to the large amount of Web data we process and provide us with timely reporting? 
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APPENDIX E: COMMON APPROACHES TO IT ROI 


Technique 

General Definition 

How to Calculate 

Advantage 

Disadvantage 

Return on 

Investment 

(ROI) 

A catchall phrase commonly 
used for several ways to measure 
business value of a project. ROI 
means profit divided by invest¬ 
ment, expressed as a percentage. 
As the numerator, profit can be 
replaced by cost reductions or 
productivity gains derived from 
the operational improvements an 
IT project yields 

Revenue or cost savings 
divided by investment 

Best applied to projects 
where all costs that will 
be incurred or all cost 
reductions that will be 
realized are known 
ahead of time, usually 
from experience on a 
similar project. 

Difficult to apply to 
entrepreneurial IT 

projects that are 

designed to help launch 
new products, services or 
businesses that translate 
to new sources of 
revenue and profits. ROI 
doesn't consider risk, 
flexibility & intangibles. 

Net Present 
Value (NPV) 

Refers to the future net cash 
flow a project is expected to 
deliver, minus the investment. It 
defines the value of a project in 
"today's dollars." The calculation 
is based on the company's cost 
of capital used for assessing 
proposal alternatives. It returns a 
nominal amount. 

Cash inflow minus cash 
outflows calculated in today's 
dollars. 

Includes all cash flow 
related to a project. 
Considers the time value 
of money, or the 
difference in the value of 
a dollar today and what 
it might be three years 
from now. 

The highest NPV doesn't 
always correspond to the 
most efficient use of a 
company's capital. 

Internal Rate 
of Return 

(IRR) 

One of several metrics that 
considers the time value of 
money, IRR expresses the dollar 
returns expected from a project 
as an interest rate. Once the rate 
is established, it can be 
compared to rates earned by 
investing in other projects. More 
informally, IRR is also known as 
the "hurdle rate" because it's 
usually the lowest rate of return 
that management will accept. 
Typically, a project must earn an 
IRR that is several percentage 
points higher than the cost of 
borrowing, to compensate the 
company for its risk exposure 
and time. 

C= all costs associated with 
the project and call it. 
R=estimate of all returns 
resulting from the project. 

T= how many months or years 
company will realize returns. 
i= firm's minimum acceptable 
rate of return 

Calculate the interest rate: 

C=R X T (i). 

Reference a NPV chart listing 
the value of a $1 annuity and 
find the corresponding interest 
rate. Compare that interest 
rate to the minimum 
acceptable rate and determine 
if project will leap over hurdle 
rate. 

Includes all cash flow 
related to a project. 
Considers the time value 
of money. It enables the 
comparison of rates of 
return on alternative 
investment options. 

Given two investment 
alternatives and 

assuming that both fit 
strategic objectives of 
the organization, the 
investment with higher 
internal rate of return 
should be selected. 
Conceptually it is the 
easiest method to 

understand. 

Disadvantages: Assumes 
cash flows are reinvested 
at the IRR. Cumbersome 
to calculate interest rate 
when cash flows vary 
widely year to year. 
There is no specific 
formula that can be used 
to calculate the IRR; it is 
found by interpolation. 

Payback 

Period 

How long it will take an 
investment to pay for itself 

Initial project investment 
divided by cash inflows (or 
cost reductions) per year. 

It's simple and 

understandable. 

Time value of money 
and cash or other 
benefits received after 
payback period are not 
recognized, which 

determines profit. 

Economic 
Value Added 
(EVA) 

Measures a corporation's true 
economic profit. The idea is to 
understand which business units 
best leverage their assets to 
generate returns and maximize 
shareholder value. 

Net operating profit minus an 
appropriate charge for the 
opportunity cost of all capital 
invested in an org. - 
EVA= Net Operating Profit 
After Taxes (NOPAT) 
(Capital X Cost of Capital) 

Can more precisely 
define value in terms 
specific to an enterprise. 

Complex, proprietary 

(expensive) and not 
widely used. Metric is 
extraordin-arily dependent 
on the size of a business. 
Big operations/ projects 
tend to produce big EVAs, 
while small operations/ 
projects are much smaller. 


Table 20, Common Approaches to IT ROI 
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APPENDIX F: SPREADSHEET OF ROI RETURNS 



Weekly 

Weekly 

Bi-Weekly 

Bi-Weekly 

Mouthly 

Mouthly 

Quarterly 

Quarterly 

Variable Calculated 

Best Case 

Worst Case 

Best Case Worst Case 

Best Case 

Worst Case 

Best Case 

Worst Case 

Number of Procedures 

No. procedures required to perform 
activity in a non-optimized portal 

50 

50 

110 

110 

310 

310 

1200 

1200 

No. procedures required to perform 
activity in an optimized portal 

45 

48 

95 

102 

285 

300 

1100 

1150 

Number of Hours 

No. hours required to complete activity in a 
non-optimized portal 

30 

30 

68 

68 

125 

125 

400 

400 

No. hours required to complete activity in 
an optimized portal 

Number of Knowledge Workers that 

26 

28.5 

57 

64 

110 

116.5 

350 

375 

Conduct same Task 

4000 

3500 

1000 

900 

500 

450 

250 

220 

Cost of Activity (direct and Overhead) 
Additional Cost (Associated to the optimized 

700 

700 

1200 

1200 

2400 

2400 

15000 

15000 

portal for this activity) 

200 

300 

400 

700 

800 

1100 

1000 

1400 

Indirect Costs for Non-Optimized Portal 

Indirect cost (time in minutes) assigned 
procedures in non-optimized portal 

36 

36 

37.0909 

37.09090 

24.19354 

24.19354 

20 

20 

Average cost assigned non-optimized portal 
over period to complete activity 

Indirect Costs for Optimized Portal 

14 

14 

10.90909 

10.90909 

7.74193 

7.74193 

12.5 

12.5 

Indirect cost (time in minutes) assigned 
procedures in optimized portal 

34.66666 

35.625 

36 

37.64705 

23.15789 

23.3 

19.09090 

19.56521 

Average cost assigned optimized portal 
over time period to complete activity 

Amortized Indirect Costs Optimized Portal 

17.92592 

20.10416 

14.79876 

17.93540 

10.21754 

11.12266 

12.84090 

13.44565 

Average cost assigned optimized portal 
amortized over one year time period 

13.57407 

13.98437 

10.76367 

11.35861 

7.64444 

7.76155 

12.15909 

12.53260 

Frequency of Activity (answer only one) 

48 

48 

24 

24 

12 

12 

4 

4 

Procedures Gaiued iu Oue Activity Period 
Perceutage Procedures Gaiued iu Oue 

6.923076 

2.526315 

18.333333 

6.375 

38.863636 

21.88841 

157.14285 

76.66666 

Activity Period 

Procedures Gaiued iu Oue Year for this 

3.84615 

1.052631 

3.030303 

-1.477272 

4.47214 

3.83497 

4.76190 

2.22222 

Activity 

92.30769 

25.26315 

80 

-39 

166.36363 

142.66094 

228.5714 

106.6666 

Perceutage Chauge iu Procedure 

Productivity for Activity iu Oue Year (%) 
Total Cost of Procedures for this Activity iu 

3.846153 

1.052631 

3.030303 

-1.477272 

4.472140 

3.834971 

4.76190 

2.22222 

Oue Year 

33830.76 

33915.78 

29277.192 

29543.75 

29709.090 

29980.257 

61142.857 

61493.333 


Saviugs of Procedures for Activity iu 
Optimized Portal over Oue Year 

1061.538 

37.89473 

395.53429 

-1169.204 

378.88563 

-75.78568 

1714.28571 

-160 

Perceutage Saviugs of Procedures iu 
Optimized Portal over Oue Year 

3.140646 

0.111783 

1.354569 

-3.963405 

1.28001 

-0.253463 

2.810304 

-0.260586 

Auuual Saviugs to Euterprise for Activity 
(if more than one employee) 

Percentage Annual Savings to Enterprise 

4246153.85 

132631.57 

395534.29 

-1052284.09 

189442.81 

-34103.55 

428571.43 

-35200 

for ActivMr 

3.15933 

0.112781 

1.373363 

-4.05962 

1.31550 

-0.26312 

2.85695 

-0.26663839 

Additional Similar Activities Accomplished 
across Enterprise 

Percentage Additional Similar Activities 

8205.128 

1842.105 

842.1052 

-344.11764 

291.86602 

213.99141 

51.94805 

20.40579 

Accomplished across Enterprise 

Virtual K-Workers Gained for Aetivity 

4.273504 

1.096491 

3.50877 

-1.5931372 

4.86443 

3.962804 

5.194805 

2.318840 

across Enterprise 

Percentage Virtual K-Workers Gained for 

170.9401 

38.37719 

35.0877 

-14.338235 

24.32216 

17.83261 

12.98701 

5.10144 

Activity across Enterprise 

4.273504 

1.096491 

3.50877 

-1.593137 

4.86443 

3.962804 

5.194805 

2.318840 

Table 21, Comparative ROI 

Returns for Different Activities: One Year 
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Weekly 

Weekly 

Bi-Weekly 

Bi-Weekly 

Monthly 

Monthly 

Quarterly 

Quarterly 

Variable Calculated 

Best Case 

Worst Case 

Best Case 

Worst Case 

Best Case 

Worst Case 

Best Case 

Worst Case 

Number of Procedures 

No. procedures required to perform 
activity in a non-optimized portal 

50 

50 

110 

no 

310 

310 

1200 

1200 

No. procedures required to perform 
activity in an optimized portal 

45 

48 

95 

102 

285 

300 

1100 

1150 

Number of Hours 

No. hours required to complete activity in a 
non-optimized portal 

30 

30 

68 

68 

125 

125 

400 

400 

No. hours required to complete activity in 
an optimized portal 

Number of Knowledge Workers that 

26 

28.5 

57 

64 

no 

116.5 

350 

375 

Conduct same Task 

4000 

3500 

1000 

900 

500 

450 

250 

220 

Cost of Activity (direct and Overhead) 
Additional Cost (Associated to the optimized 

700 

700 

1200 

1200 

2400 

2400 

15000 

15000 

portal for this activity) 

200 

300 

400 

700 

800 

1100 

1000 

1400 

Indirect Costs for Non-Optimized Portal 

Indirect cost (time in minutes) assigned 
procedures in non-optimized portal 

36 

36 

37.09090 

37.0909 

24.19354 

24.19354 

20 

20 

Average cost assigned non-optimized portal 
over period to complete activity 

Indirect Costs for Optimized Portal 

14 

14 

10.90909 

10.9090 

7.741935 

7.741935 

12.5 

12.5 

Indirect cost (time in minutes) assigned 
procedures in optimized portal 

34.66666 

35.625 

36 

37.64705 

23.15789 

23.3 

19.09090 

19.5652 

Average cost assigned optimized portal 
over time period to complete activity 

Amortized Indirect Costs for Optimized 
Portal 

17.92592 

20.10416 

14.79876 

17.93540 

10.21754 

11.12266 

12.84090 

13.44565 

Average cost assigned optimized portal 
amortized over one year time period 

13.52777 

13.91927 

10.67595 

11.21563 

7.52748 

7.608777 

12.04545 

12.38043 

Frequency of Activity (answer only one 
choice below) 

96 

96 

48 

48 

24 

24 

8 

8 

Procedures Gained in One Activity Period 
Percentage Procedures Gained in One 

6.92307 

2.5263 

18.3333 

6.375 

38.8636 

21.8884 

157.1428 

76.6666 

Activity Period 

Procedures Gained in One Year for this 

3.84615 

1.05263 

3.0303 

-1.47727 

4.47214 

3.83497 

4.761904 

2.2222 

Activity 

184.6153 

50.5263 

160 

-78 

332.7272 

285.321 

457.1428 

213.33 

Percentage Change in Procedure 

Productivity for Activity in One Year (%) 
Total Cost of Procedures for this Activity in 

3.84615 

1.05263 

3.0303 

-1.47727 

4.472140 

3.83497 

4.761904 

2.2222 

One Year 

67430.76 

67515.78 

58077.19 

58343.75 

58509.0909 

58780.257 

121142.857 

121493.33 


Savings of Procedures for Activity in 
Optimized Portal over One Year 

Percentage Savings of Procedures in 
Optimized Portal over One Year 

2353.846 

3.492353 

391.5789 

0.580116 

1268.261 

2.186657 

-1594.659 

-2.735264 

1666.862 

2.854216 

1028.686 

1.752446 

4571.428 

3.77804 

1173.333 

0.966501 

Annual Savings to Enterprise for Activity 

(if more than one employee) 

9415384.62 

1370526.31 1268261.563 

-1435193.18 833431.085 462908.7637 

1142857.1 

258133.33J 

Percentage Annual Savings to Enterprise 
for Aetivilr 

3.50274 

0.582706024 2.201827701 

-2.76846 

2.893777 

1.785837 

3.8093 

0.97772 


Additional Similar Activities Accomplished 


across Enterprise 

16410.25 

3684.210 

1684.210 

-688.2352 

583.7320 

427.9828 

103.8961 

40.8115 

Percentage Additional Similar Activities 
Accomplished across Enterprise 

4.273504 

1.096491 

3.50877 

-1.593137 

4.864433 

3.962804 

5.194805 

2.31884 

Virtual K-Workers Gained for Activity 
across Enterprise 

170.9401 

38.37719 

35.0877 

-14.33823 

24.32216 

17.83261 

12.98701 

5.101449 

Percentage Virtual K-Workers Gained for 
Activity across Enterprise 

4.273504 

1.096491 

3.50877 

-1.593137 

4.864433 

3.962804 

5.194805 

2.31884 


Table 22, Comparative ROI Returns for Different Activities: Two Years 
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1 Variable Calculated Best Case _ Worst Case. _ ^_ Difference , 


3 

Number of Procedures 

No. pioceduies required to peffoim edivityin anon-optimixed portal 












50 





50 







No. procedures requited to perform activity in «n optiissedportal 

45 





48 





-3 

5 iNiutiber of Hours 













No. hours required to complete activity in a non-optimized portal 

30 

108000* 




30 

108000* 





7 

No. hours required to complete activity in an optimized portal 

26 

93600* 




28.5 

102600 




-2.5 


Number of Knowledge Workers that Conduct same Task 

4000 





3500 





-500 


Cost of Activity (direct and Overhead) 

700 





700 






10 

11 

Additional CostfAssociatedto the optifflizedDortal for this activity’) 

200 





300 





-100 

bidirect Costs for Non-Optimized Portal 












12 

Indirect cost (time in minutes) assigned procedures in non-optimizedportal 

3^ 

2160* 

0.003462963' 



36' 

2160' 

0.000462963' 




13 

Average cost assigned non-optimizedportal over period to complete activity 

14 

0.071428571' 




14 

0.071428571' 





14 

bidirect Costs for Optimized Portal 












15 

Indirect cost (time in minutas) assigned procedures in optimized portal 

34.66666667' 

2080* 

0.000480769' 



35.625* 

2137.5' 

01)00467836' 



-57.5 

16 

Average cost assigned optimized portal over time period to complete activity 

17.92592593' 

0.055785124 

m.666666f 

606.666666f 

0.86666666? 

20.1041666? 

0.0^740933' 

96? 

66? 

0.9? 

-2.178240741 

17 

Amortized Indirect Costs for Optimized Portal 


0.07366985* 










18 

Average cost assigned optimizedportal amortized over one year time period 

13.57407407' 

610.8333333' 

4.166666667' 


13984375' 

0.07150838' 

67125' 

6.2? 


0 41030)926 

19 

Frequency of Activity(answer only one choice belot^ 

48* 




48' 






20 

Weekly (enter the value of 1 if this applies) 

1 

48' 




1 

48' 





21 

Bi-Weekly (enter the value of 1 if this applies) 


o' 





o' 





22 

MontWy (enter the value of 1 if this qjplies) 


o' 





o' 





23 

Bi-Monthly (enter the value of 1 if this ^plias) 


o' 





o' 





24 

Quarterly (enter the value of 1 if this ^plies) 


o' 





o' 





25 

Semi-Annually(entetthe value of 1 if this ^hes) 


o' 





o' 





26 

28 

AmuallT ('enter the value of 1 if this anoHesT 


o' 





o' 





Procedures Gained in One Activity Period 

6.923076923' 

5192307692* 




2.526315789' 

X.52631579' 




-4,396761134 

29 

Percentage Procedures Gained in One Activity Period 

3.84615384(? 

0038461538' 

1.038461538' 



1.052631579' 

0010526316' 

11)10526316' 



-2.793522267 

30 

Procedures Gained in One Year for this Activity 

92.30769231' 

' 2400* 

2492.307692* 



25.26315789' 

' 2403 

2425.263158' 



' -67.04453441 

31 

Percentage Change in Procedure Productivity for Activity in One Year CHi) 

3.84615384^ 

0.038461538' 

1.038461538' 



1.052631579' 

0.01052631^ 

1.01052631? 



-2.793522267 

32 

J4 

Total Cost ofProcedures for this Activity in One Year 

33830.76923' 

2492.307692' 




33915.7894? 

2425.263158' 




85.02024291 






35 

Savings ofProcedures for Activity in Optimized Portal over One Year 

1061.538462 

233.7692308 

33600 

1292.307692 

37.89473684 

315.789473? 

33600 

353.684210? 


-1061.538462 

36 

Percentage Savings ofProcedures in Optimized Portal over One Year 

3.140646336' 

0.031406463' 

33800' 

0.111783884 

0.001117839' 

33900' 



-3.028862451 

37 

Annual Savings to Enteiprise for Activity (if more than one employee) 

' 4246153^' 





'1326315789' 





-4113522267 

;-IH 

Percentage Annual Savings to Ejdetprise for Activit}' 

3.15933lf 

01)3159336' 

134400200 

134400000 


0.11278166? 

0.00112781? 

117600330 

117600000 


-3.046554291 














40 

Additional Similar Activities Acconqilished across Enterprise 

8205.128205' 

369233.7692* 




1842.105263' 

88421.05263 




-6363.02SI42 

41 

Percentage Additional Similar Activities Acconqilished across Enterprise 

4.273504274 

0.042735043' 

1.04273X143' 

200205.128? 

192000' 

1.096491228' 

0.01096^12' 

1.010964912' 

169842.1053' 

168000* 

-3.177013)45 

42 

Virtual K-Workers Gained for ActiviQ' across Enterprise 

'’1709401709' 





' 38.37719298' 





-132,562978 

43 

Percentage Wrtual K-Workers Gained for Activity across Enterprise 

4.273504274 

0.042735043* 

1.0427350^' 

4170.940171* 


1.096491228' 

0.01096^12' 

1.01096^12' 

3538377193' 


-3.17701X45 

44 













45 













46 













47 













48 













49 













SO 













51 













52 













53 
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56 
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Table 23, Expanded Field view of Comparative ROI Weekly Returns 
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Grant A. Jacoby: 


Grant A. Jacoby; 

Normalzed Average oF 


Tot^ Number oF 
Procedures divided by 
Total Cost! (B or 6 3) / (0 


Grant A. Jacoby: 

Normailieed Average oF 
Cost per Procdure; 1 / (C 


Mnutes For every one 


0.071^8^ 


Procdure; (C or H 15) / 


Grant A. Jacoby: 


34.6666666'j 

17.92592593 


Number of Procedures 
per Seconds For work 
requr^: (6 or G4) / (01 
G 6) X 3600 seconds 


Normalized Average oF 
Seconds for every one 
Procdure: 1 /t P or I IS , 


005578512^ 


l6^667 


14666667 


0.0736698' 


■year tune period 


Grant A. Jacoby: 

Ratio oF Hours to 
complete an activity 
between optimized and 
rion-optirrdzed portal; (B 
or G 7>; (B or G 6) 


Grant A. Jacoby: 

Frequency of Activity i 


Frequency of Activity in 
One Year 


Grant A. Jacoby: 

Total Cost of optimized 
poital to complete 
artlvity;(B orGi0)x(F 
Of K 16) ratio 


Grant A. Jacoby: 

Total Cost of optirrnzed 
portal to complete 
activity plus the addition 
of the cost associated to 
enhac^ it through 


Grant A. Jacoby: 

Total Numter of 
Rcgc^ures divided by 
number 

of P«Sedi^e?^r'dd|gn 
B or G 3) 16) 


Nunber of Minutes per 
Procedure completed: 

1 / (C or H 16) 


&ant A. Jacoby: 

Frequency of Activity i 
Cne Year 


Grant A. Jacoby: 


Frequency of AcBvity i 


Frequency of Activfty 
Provided from C or H 20 


j Grant A. Jacoby: 

Frequency of Activity i 


6.923076923' 


jl.92307(»2] 


ftequerviy cf Activity i 


3.84615' 


Grant A. Jacoby: 

Total Amortized Cost cf 
Total Costs, I.e., Cost of 
Activity and Additional 
Metrics Cost; (E or J 16) 
+ (E or J 18) 


Grant A. 'Jacoby: 

Number of ProcMiii;es 
required to complete^ 
Activity in optimized 
portal divided by Total 
Amorbzed Costs: (B or 
4)/(Dor 118) 


Procdure to compile 


Activity in optimized 


tnityin Optimized Portal over One 
lures in Optimized Portal overOiie Year 
for Acti!rity(ifmoietlisriode employee) 


1292307692'' 


Addition^ Cost 


associated to metrics 


11593^1 


Grant A. Jacoby: 

Total number of seconds 
required to complete 
Acti^ln non-opttnized 
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APPENDIX G: GLOSSARY _ 

-Quantitative -Qualitative - Both 

Accessibility : Information is considered accessible when it is available, reachable, and 
understandable. (Assuming the information exists, ideally availability could occur from 
any point of access of and point of entry to the information system. Although how 
reachable information is depends on a number of factors, in this context the more 
information is compatible with a shared Information Architecture, the easier it is to 
retrieve. How this information is then packaged and presented in a fashion that can be 
understood by the knowledge worker is the last stage of accessibility.) 

(Note: Accessibility also has another meaning in terms of access or permissions to get 
information.) 

Additive : The term that is characterized by the addition of independent statistical sums 
that are representative of the sum of their parts when summed as a whole. For example, 
when the total page views for each page in a yrppt, such as /office, are summed by simple 
addition that sum correctly represents the total page views for that vroot. Antonym non¬ 
additive 

Associative relationship : A relationship between items that are closely related 
conceptually but not hierarchically and are not members of an equivalence set. 

Attempted Downloads : The number of attempted downloads as indicated by the HTTP 
Status for acknowledged page request. See also Successful Downloads . 

Average minutes per usage day : The average number of minutes spent on the website, 
category, channel, or application during the day, per visiting person. 

Average minutes per usage month : The average total number of minutes spent on the 
website, category, channel, or application during the month, per visiting person. 

Average minutes spent per unique page : The average number of minutes spent on each 
unique page during the day. 

Average unique pages per visitor in a month : The average number of different pages 
viewed per day over the course of the month by those persons visiting the domain, global 
domain, property, or category. A unique page is defined as a specific URL that was 
successfully loaded by the browser at least once in the day. Even if the page was viewed 
many times, the page is counted only once under the unique page definition. In the case 
of non-Web content, unique pages are the equivalent of unique window titles, if 
applicable, or if not, are undefined and set to zero. The multiple URLs displayed within a 
framed page are not counted as additional unique pages; rather, all credit is given 
exclusively to the host URL, which does not change as users update constituents of the 
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page. Therefore, websites utilizing frames may show lower unique page estimates than 
websites that do not utilize frames. 

Average usage days per visitor per month (Frequency) : The average number of different 
days in the month, per person, in which a website, category, channel, or application was 
visited. A person is defined as a visitor if they access at least one "page" of content within 
the website, category, channel, or application. A day is defined as 12:00 midnight 
through 11:59:59 PM. 

Authoritative Flags : An indication that the information presented has gone through a 
screening process to discern its credibility, i.e., confirmation that information is timely 
and trustworthy. 

Best Bets : Results from an authoritative Intranet sponsored site maintained by an internal 
division or group that is regularly updated and contains large amounts of information 
relevant to the search topic and does not require any special security permission to view 
it. 

Best Practices : Effective means of doing work in a preferred way that takes place and 
changes over a period of time based originally from guidelines. 

Clickstream Analysis: A Web analytics software that tracks and measures all visitor 
behavior (mouse movement and links invoked) online and offline both quantitatively and 
qualitatively to assess user behavior patterns. 

Cluster : A group of servers. Specifically, a cluster is a group of servers assigned to one 
sub-domain of an enterprise. Synonym: Domain . 

Cognitive Factor : The critical analysis ability of an individual. For example, in Web 
analytics should attempt to account for the cognitive ability of the knowledge worker to 
further link associated information and to take into account additional commonalities and 
solutions that are not readily apparent or realizable by quantitative measurements alone. 

Communications : Human and technological systems used for sending and receiving 
messages. In the case of exercising Web analytics, communications is what supports the 
engagement of the knowledge worker. 

Communication of Authoritativeness and Importance : Communicating to the knowledge 
worker the credibility of an information system within that system to ensure confidence 
and trust. See: Authoritative Flags. 

Communication of Understanding : Communicating to the knowledge worker meaning 
and significance to ensure acceptance and engagement, which further increases the 
audience’s propensity to act on information in a common way, i.e., a Glossary of Terms 
and Definitions, Mouse-over, and best practices. 
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Community : Enterprise communities are based on groups of shared interest. A 
community can be a group or a task, but they both share a common interest. 

Composition : The percentage of a website, category, channel, or application's visitors 
that belong to a specific demographic grouping. 

Content and Content Properties : Content - All documents — help text, search results, 
forms, or application information — that is delivered to an knowledge worker via a 
website, or application. It is both subject and specific information. For example, 
“People” is a subject and a person by name is specific information. Content is organized 
and found depending on its properties and associations 

Content Properties - How content is described in order to affect the associations made in 
information retrieval and presentation. The characteristics of a content item make up its 
properties, such properties (a.k.a. attributes) include author, length, name, etc. 

Contextualization : The process of placing an idea, information item or knowledge worker 
in a relevant environment to enhance understanding. See: Personalization and 
Customization. 

Customization : Ability for knowledge workers to self-configure the contents/constituents 
of their domain site. In essence, individuals are allowed to subscribe to constituents, or 
sub- constituents of a portal, including navigation, content, search, etc., in order to 
provide and store information (i.e., like the notion of “my saved searches”) most relevant 
to their interests, position or role. 

Data : Information, often in the form of facts or figures (for example, numbers, text, 
images, and sounds) obtained from experiments or surveys, in a form that is suitable for 
storage in or processing by a computer used as a basis for making calculations or drawing 
conclusions. 

Data Mining : Analysis of data in a database using tools which look for trends or 
anomalies without knowledge of the meaning of the data. 

Data Warehouse : For any Web analytics, the system for storing Web analytic data for the 
enterprise Web servers. 

“Discoverability” : - Facilitate information discovery through presentation of relative 
associations which lead to better decisions, increased productivity and effectiveness that 
would not have been likely otherwise. Discoverability requires three events to happen: 1) 
associative connections between content items are created; 2) the information/ knowledge 
is presented to illustrate relative associations that can be acted upon; and 3) the 
knowledge worker has the cognitive ability/recognizes there is an action to be taken. 

Digital media reach percentage : The percentage of projected individuals who visited a 
specific website, category, channel, or application among the total number of projected 
individuals using any digital media during the course of the reporting period. 
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Domain : This term is used to identify the domain of an enterprise. See also: Sub-domain . 


Domain Integration Framework : Representation of the relationships/strueture between 
the key elements within a design. It shows how these elements interact and must 
“transfer” information, so that the architect can build the environment. Examples are 
information maps (document elements), server topography, scope of services, 
permissions (ownership). 

Download : Any file that is sent to a client by an HTTP server with selected extension, 
such as .exe, .doc, .zip, etc. 

Drilling : To analyze data in more depth by moving down or up levels within a category 
or classification, to include date hierarchies. An example would be analyzing page views 
for June 2000, then "drilling-down” to June 1, 2000. 

Editorial Programming : The editor’s experience should be a straightforward and 
consistent use of accepted best practices and best tools available (from within the 
company) to organize, access and re-use information in a desired manner to produce a 
desired outcome. 

Engagement : The more utility a system provides the more recognized and compelling it 
becomes. In other words, there is a gradual synergy that manifests from enthusiastic 
knowledge workers through escalating levels of commitment up to outright ownership as 
a result of perceived usefulness of a system/service. 

Equivalence relationship : A relationship between items that is conceptually equivalent. 
Ex: ASP = Active Server Pages 

External Referrer : In contrast to internal referrer, external referrers are the websites 
which are not managed by the enterprise, and are therefore generally not tracked. 
Examples are Yahoo, Lycos, Excite, etc or a business partner domain. 

Extranet : An Extranet is somewhat similar to an Intranet . Extranets are designed 
specifically to give external limited access to certain files using the Internet protocol and 
the public telecommunication system to securely share part of a business's information or 
operations with suppliers, vendors, partners, customers, or other businesses. 

Finding : This is a process employed by knowledge workers to seek the information they 
need to do their jobs. 

QUID : QUID stands for Global Unique Identifier, a unique 32-character hexadecimal 
string which is generated for each user who accepts cookies. 

GUIDed User : A user assigned a GUID in a cookie. Also referred to as GUIDs and 
users. 
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Guideline : Guidance on how to meet established standards. Over time guidelines 
become best practices. 

Hierarchical relationship : A relationship between items based on one item being a part of 
another, one item being an instance of another, or one item being descended from 
another. This creates linkages from broader to narrower, generic to specific or parent to 
child. 

Hit : In Web activity, a hit is registered when a file is sent to a client by an HTTP server. 
This includes graphics, HTML, EXEs, etc. 

HTTP Status Codes : The code associated with a page request that is recorded in the IIS 
log for each HTTP method exchange between client and server. Eor example, a 
successful page request results in an HTTP code 200 or OK and can be imported into a 
log file for the data warehouse. Occasionally there is a 404 page error which occurs 
when a server has not found anything matching the request URL (no indication is given 
on duration of condition). 

Index : The composition of the website, category, channel, or application compared to the 
demographic composition of the universe. 

Information : The meaning of data as it is intended to be interpreted by people. Data 
consists of facts, which become information when they are seen in context and convey 
meaning to people. 

Information Architecture : The information architecture (lA) of a corporation is the sum 
and organization of all its data, taxonomies, tools and products. lA for Web analytics 
needs to encompass not all the lA that is available but only the best elements of these 
groups which can be further developed and integrated to improve control of content and 
context to meet knowledge workers needs and to exceed their expectations as well. 

Internet : The Internet is a public cooperative of networks and gateways around the world 
that uses a portion of the total resources of the currently existing public 
telecommunication networks that use of a set of protocols called TCP/IP (for 
Transmission Control Protocol/Intemet Protocol. Two recent adaptations of Internet 
technology, the intranet and the extranet, also make use of the TCP/IP protocol. 


Intranet : An Intranet is a private, secured information portal designed specifically for the 
internal communications of small, medium or large businesses, enterprises, governments, 
industries or financial institutions of any size or complexity. Intranets can be custom- 
designed to fit the exact needs of businesses no matter where they are situated. Users, 
also referred to as knowledge workers, of Intranets consists mainly of company 
employees and business partners. 
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Internet Architecture : All constituents used to provide a means of access or transfer to a 
point of entry. 

Information Grouping & Segmenting : Information grouping is the logical collection of 
similar and relevant information, such as Content Nuggets and Categories respectively. 
Another aspect of this is information segmenting in which only the relevant parts of a 
document, i.e. a paragraph or two are extracted and placed into fragmented groupings 
instead of the entire document. For example, first 200 words of a document returned as 
part of a search result. 

Information Life Cycle : The events often repeated again and again in maintaining the 
relevance and accessibility of content in an information system. These events include 
updating, versioning, archiving and, when necessary, deleting. See; Retention 

Information System : A procedure that combines and organizes related elements into a 
complex whole to achieve meaningful representation of data. 

Internal Referrer : In contrast to external referrer, internal referrers are the websites 
managed within an enterprise Intranet. 

No Referrer Includes the following ways a user can enter a website; 

• favorites or bookmarks 

• a manually typed URL 

• embedded li nks in documents or e-mail messages 

• default start page of a browser 

• use of a browser configured not to pass headers 

• server site redirect 

Knowledge : Understanding gained through experience or study 

Knowledge Worker: One who gathers, analyzes, adds value and communicates 
information to empower decision-making. (The nature of ‘k-work’ is ad hoc, demand- 
driven and creative, both in the ability to create new knowledge greater than the sum of 
its parts and in the ability to present the knowledge in a highly communicative way). 

Log Availability : An indicator as to the whether log files are present and ready for 
importation by analytics team to be aggregated into the data warehouse. Factor affecting 
log availability includes network anomalies, server outages, and corrupt disks. 

Metadata : Information about a content item derived automatically or by human analysis. 
Types of metadata include; administrative (managing and administering content), 
descriptive (describing/identifying) and technical (related to how a system functions of 
how metadata behaves). Metadata can be expressed in name/value pairs. See; Surrogate 
Ex: author ; Jane Doe 

UID ; 7395ZX32Y2001 
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Metadata Schema : Accumulation of metadata representing classes of information 
common to all members of a given content sent. Metadata schema is part of an overall 
schema. See; Schema 

Metrics : Though primarily numeric, mainly quantitative and some qualitative facts that 
are associated with Web usage, such as the count of unique users for a specific page on a 
given day. 

Minutes per Page View : Total minutes visitors spent on the sites divided by total page 
views. 

Mean Time Between Failure (MTBF) : MTBF measures how long a server runs before it 
"fails" or needs to be rebooted. 

Navigation : Method of moving through the domain framework by way visual 

presentation and consistent choices. Navigation has 2 basic types: local (also known as 
vertical) and global (also known as horizontal). 

Local navimtion presents choices leading to subtopics or sub-areas of a site, usually 
defined by one of its main menu subjects (often referred to as a “drill down”). 

Global navisation presents choices leading to other main areas of a site, i.e. Home; 
Search; About. Global navigation is consistent throughout an information system which 
allows knowledge workers to go across portals with a sense of familiarity. 

Non-additive : The term that refers to not having a numerical value that is equal to the 
sum of the parts. In Web analytics, this term is used in application of the summation of 
unique users and unique pages. 

Null User : A user whose browser cannot accept a cookie, or a user who has chosen not 
to accept a cookie, or a user who has not been to a site on which cookies are issued. We 
cannot assign a permanent QUID to a null user and therefore cannot track their activity 
on a site. 

Online User : All users who come to an enterprise Intranet. In general, we can tell what 
type of browser customers are using, where they came from, and what areas of the site 
they visit. 

Page Instrumentation : The concept of adding "trigger” code in the form of scripts to a 
Web page that is able to relay user interactions to a service that archives the activity for 
analysis. Synonym; Click Stream Data Collection . 

Page Views : Any file sent to a client that provides information, such as http, ASP, Word 
documents, etc. Depending on the business, the following files may or may not be 
counted as page views; avi, bmp, cdf, class, css, dat, dll, gif, idq, inc, jpg, js, toe, and 
wav. 


157 
















Page Views per User : Page views by GUIDed users divided by the number of unique 
users (GUIDed). We use this number to indicate how many pages are viewed by a user 
on average. 

Page Views by GUIDed : Users The count of page views by GUIDed users only. 

Percentage Page Views by GUIDed : Users Page Views by GUIDed users divided by 
Total page views. For example, if there are 10,000 pages viewed under a vroot, with 
3,000 pages viewed by GUIDed users, this means that GUIDed users account for 30% of 
the page views, and non-GUIDed account for 70% of the page views. 

Persona : If a business fails to understand its users, then it will probably create a poor 
product or service. Personas (or User Archetypes) are a way of depicting the users of 
products, such as user behavior while visiting portal sites. Using personas allows 
development teams to focus more on design and apply appropriate effort on the right set 
of features to support these users. Named and developed by usability engineers, personas 
are tools that help make customers very real to the product developers while comprising a 
rich collection of consumer data, field research and other studies. Key objectives are to 
ensure products are both useful, so they do the things people want them to do, and usable, 
so people can accomplish those things easily. One of the most important functions of 
usability engineers is to communicate to the development team, as clearly as possible, 
who the users really are. When designers and developers don't share a distinct image of 
their user, they carry different interpretations of 'user' around with them. As a result, they 
are prone to developing a schizophrenic interface or - taken as a whole across a large 
enterprise - very inconsistent interfaces that leave for a bumpy, if not dissatisfying, surf 
experience from portal to portal. 

Personalization : A method of contextualizing information for a knowledge worker based 
on what is known about the knowledge worker. A site can use personalization to alter 
navigation and content presented according to the perceived needs of the knowledge 
worker. Examples include content filtering based on role; authentication based on name 
or status as a manager. 

Pivot Table : An interactive data report that permits the user to manipulate the hierarchy 
of data filters as well as the organization of column and row headings. This provides a 
means to rearrange data for different views during analysis. 

Point of Access : Content can be reached through a variety of points of access. These 
access points reside within the content item itself, i.e. full text, or are associated with the 
content through its properties and relative associations. (Compare to the use of the term 
in identifying physical points of access to an information system such as desktop 
computers, hand-held computers, and wearable computers.) 

Point of Entry : Eogical and visually identifiable place on an information system where a 
knowledge worker can begin to seek information. This point of entry is then accountable 
for the knowledge worker’s experience. 
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Policy : Accepted and codified set of decisions designed to aoeomplish identified 
business goals. Any and all Web analytie polieies coordinated by a Business Intelligence 
team should reflect business values. 

Portal : A means into an information system, such as a corporate intranet, established 
over a eoherent body of information or community of interest, also known as a site. This 
(personalized) Website is built on a common set of application services that enables firms 
to conneet employees, customers, partners, and suppliers to all appropriate eorporate 
computing resources from any Internet-eapable device. 

Produetivity : The better knowledge workers apply knowledge, the more productive they 
are. Therefore, how this knowledge is managed for and by knowledge workers is the 
most integral, inextrieable aspect of improving knowledge worker productivity. 
Productivity benefits from improved knowledge management include a eonservation of 
resources, maintaining eompetitive edge and ereating new opportunities. 

Profiled User : Is a GUIDed user who has submitted profile data through a RegSys 
applieation. An un-profiled user is a GUIDed user who has not registered. 

Query manipulation : The process of modifying an original query to best match the 
teehnology, eontent and knowledge worker behavior to improve relevaney of information 
presented. Query manipulation ean be done manually by the knowledge worker, 
interaetively with the knowledge worker and the information system working together or 
automatically by the search technology. Query manipulation includes such things as 
word breaking and expanding a query term to include all of its synonyms. 

Raw Logs : Log files preserved in the state as found on their Web server of origin. An 
analyties application processes raw logs using a server application, such as Commerce 
Server 2000 for example, as it ports these logs into the data warehouse. 

Reach: Total unique users for a specific cluster or vroot divided by the total number of 
unique users for the entire enterprise domain. 

Redirect : The HTTP action of sending a Web user to another physical URL upon the 
request of another. Redirects are recorded in the logs by HTTP status codes in the range 
of 300-399. 

Redirects are specific to whether they are client-side or server-side. For example, a client- 
side redirect results in another 200 status code whereas a server-side redirect results in a 
302. 

Referrer : Indicates how users get to the pages they are on and which URL a user has 
come from. See also Referrer type . 

Referrer Type : There are at least three types of referrals relevant to Web analytics: 
external referrer, internal referrer, no referrer 
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Referring Stats : This is the statistic relating to referrer in the Web analytics program. 


Relative Associations : The term relative associations encompasses all types of 
relationships between items in an information system. These relationships are based on 
such things as characteristics of the items (properties), knowledge worker tasks and 
interactions between the items. Once established, these associations can be leveraged or 
exposed in an information system to enable a knowledge worker to move from concept to 
concept. 

Retention : Percent of GUIDed users who viewed at least one page in a vroot during the 
previous calendar month and also visited at least one page in the same vroot during the 
current calendar month. All content items in an information system have value for a 
certain length of time. Determining this length of time is deciding an item’s retention. 
The length of time an item has value and is retained depends on many factors. An item’s 
retention can be recorded in metadata and made available for information management 
purposes. 

Roll-up : The summary of a path data takes before it resolves as a metric in a report. 

Schema : The representation of the tables that depict the logical organization of the 
database or data warehouse. It is the architecture of an information storage and retrieval 
system, including the storage structure, taxonomy arrangement, and user-interface design 
for search and retrieval. Particularly important is the interaction between the various 
elements. The schema is the design and how the pieces work together. (Schema includes 
metadata schema.) 

Search : An application employed by knowledge workers as a tool to find through direct 
surfacing or through surfacing an obvious navigational path. 

Search Stats : Statistics that are collected specifically on search engines on an enterprise 
websites that include the search strings entered by users. 

(Smart) Tags : Constituents of the schema which identify and label metadata elements. 
There is a 1:1 relationship between the tags and the elements. Smart Tags attempt to do 
this process automatically based on intelligence 

Standard : The processes, protocols and technologies established or chosen by Business 
Intelligence team as the models to follow or use to meet policy requirements. 

Subdirectory : The collection of zero or more directories or files below a parent directory . 

Sub-domain : The term used to represent the breakouts of virtual URL addresses attached 
to the enterprise domain. Synonym: Cluster . 
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Subscription : Knowledge worker seleeted delivery of speeifie eontent to their email 
inbox. This is meant to occur at a specified interval, or as a result of an event. It would 
include speeifie content but may also encompass notification following the posting of 
related information. 

Suceessful Downloads : The number of sueeessful downloads based on the HTTP Status 
code + the Win32 Code or Unix Code indieating suecess. 

Surrogate : Aeeumulation of metadata representing elasses of information eommon to all 
members of a given content sent. Metadata sehema is part of an overall schema. See: 
Sehema 

Taxonomy : The result of identifying, ereating and naming relative assoeiations between 
items in an information system. In turn, the relative assoeiations are grouped or elassified 
into taxonomic structures based on three relationship types: hierarchieal, equivalenee and 
assoeiative. Examples are eontrolled voeabularies; metadata seheme; eategory labels. 
See: Relative Assoeiations. 

Technical Information : Relating to or specializing in techniques or subjeets of applied 
seience. 

Total Estimated Unique Users : Total Estimated Users = Total Page Views / (Page Views 
by GUIDed Users / Total GUIDed Users). This calculation is made with the assumption 
that non-GUIDed users view the same number of pages as GUIDed users. 

Total Page Views : Aggregated number of page views by all users. This includes 
GUIDed and non-GUIDed users. 

Total usage minutes : The total number of usage minutes spent at the website, eategory, 
channel, or application during the eourse of the reporting period. 

Unique GUIDed : User See Unique Users . 

Unique Pages : The count of individual pages in a vroot or cluster whieh have been 
viewed at least onee in the period being measured. 

Unique Pages per User: Total unique pages divided by the number of GUIDed users. 

Unique Users : The estimated number of different individuals (in thousands) that visited 
any eontent of a website, a eategory, a ehannel, or an applieation during the eourse of the 
reporting period. Visitors who reeeive a GUID by way of eookie upon entering portals 
within the enterprise domain/website enable analyties programs to keep track of actual 
volume of different visitors to enterprise portals on a daily, monthly, and annual basis. 
(Generally unique users represent a non-additive metrie in Web analytie programs, in that 
if unique users are analyzed through separate queries for each month in a year, the sum of 
those queries would not be a eorreet representation of the actual unique users for the year. 
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The sum of the individual months would be greater than the total unique users for the 
year.) 

Unprofiled User : An unprofiled user is a GUIDed user who has not registered online via 
a registration applieation. 

URL : A Uniform Resource Locator is an Internet or Intranet address consisting of the 
Internet protocol name, host name, and other elements, often including directory and file 
name. 

User Assistance : Help made available to the knowledge worker while using an 
information system. User assistance provides guidance on how to use the system or 
additional help in finding information sought. Examples are help in context, feedback. 
Dialog boxes, and training, real time assistance from a human intermediary via a live 
chat. 

User Data : Facts and figures a knowledge worker maintains private access to for 
knowledge retention and expansion. This is also referred to as “personalization”. 

User Experience : The relationship a knowledge worker has with the intranet and the 
information provided. 

User Profile : An enterprise profile of a user associated with an enterprise ID system 
which includes specific data entered by that user. See User Type . 

User Type : Of all the Intranet users who visit an enterprise portal, a portion of them go 
to a registration wizard and accept an ID cookie that is issued to them. These users can 
now be counted as unique users, although not much is known about them at this point. If 
the user completes the basic registration form, then system admin has their e-mail name 
and department. If the user also answers more specific profiling questions on where they 
use a computer, what their role with computers is, etc., then they can be considered 
profiled users and can be categorize as a specific type of Business User. 

Vroot : Abbreviation for Virtual Root, which is the logical root for a website on the 
Enterprise domain. Typically the default page for a website will be stored directly under 
the vroot. 

Web Service : A Web Service is a unit of application logic providing data and services to 
other applications. Applications access Web Services via ubiquitous Web protocols and 
data formats such as HTTP, XML, and SOAP, without worry about how each Web 
service is implemented. Web services combine the best aspects of component-based 
development and the Web. 
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